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2  Introduction 

This  project  is  aimed  at  improving  the  state  of  the  art  of  image-guided  and  minimally 
invasive  procedures  by  developing  a  new  generation  of  clinical  techniques  along  with  the 
computer-based  hardware  and  software  needed  for  their  implementation.  The  focus  of  the 
project  is  to  develop  physician  assist  systems  incorporating  robotics,  tracking,  and 
visualization  to  improve  the  precision  of  instrument  placement  and  manipulation  in 
minimally  invasive  procedures.  This  goal  is  accomplished  through  a  technology 
development  process  starting  with  phantom  studies,  then  proceeding  to  animal  studies, 
followed  by  the  ultimate  aim  of  human  clinical  trials. 

The  project  is  led  by  the  Imaging  Sciences  and  Information  Systems  (ISIS)  Center  of  the 
Department  of  Radiology  at  Georgetown  University.  Project  collaborators  include  the 
Department  of  Radiation  Medicine  at  Georgetown,  the  Urology  Robotics  Group  at  Johns 
Hopkins  Medical  Institutions,  the  NSF  sponsored  Engineering  Research  Center  for 
Computer  Integrated  Surgical  Systems  and  Technology  at  Johns  Hopkins  University,  and 
the  Engineering  School  at  the  Catholic  University  of  America,  as  well  as  several 
commercial  collaborators.  Commercial  collaborators  include  Kitware  Inc.,  ASI 
Instruments  Inc.,  and  Siemens  Medical  Systems.  The  funds  provided  under  the  research 
are  leveraged  with  funds  from  other  research  supporters  to  create  a  synergistic 
environment  for  advancement  of  these  techniques. 

3  Report  Body 

This  section  describes  the  research  accomplishments  associated  with  each  of  the  tasks  in 
the  statement  of  work.  A  brief  overview  will  be  given  and  the  reader  will  be  referred  to 
the  published  papers  in  the  appendix  for  more  details.  This  is  an  annual  report  and 
includes  research  performed  from  December  21,  2006  to  December  20,  2007.  The  award 
number  is  W81XWH-04- 1-0078. 

3.1  Task  1:  Needle  Driver  Robotics 

There  were  two  major  accomplishments  during  the  last  reporting  period: 

1)  The  development  of  a  prototype  system  for  robotically  assisted  ultrasound  [Ding 
2008b] 

2)  The  addition  of  a  rotating  needle  driver  to  the  existing  robotic  platform  from 
Johns  Hopkins  [Shah  2008] 


3.1.1  Accomplishment  1:  Robotically  Assisted  Ultrasound 

The  first  major  accomplishment  was  the  development  of  a  prototype  robotic  system  to 
assist  the  physician  in  minimally  invasive  ultrasound  interventions.  In  current  practice, 
the  physician  must  manually  hold  the  ultrasound  probe  in  one  hand  and  manipulate  the 
needle  with  the  other  hand,  which  can  be  challenging,  particularly  when  trying  to  target 
small  lesions.  To  assist  the  physician,  the  robot  should  not  only  be  capable  of  providing 
the  spatial  movement  needed,  but  also  be  able  to  control  the  contact  force  between  the 
ultrasound  probe  and  patient.  To  meet  these  requirements,  we  are  developing  a  prototype 
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system  based  on  an  existing  six  degree  of  freedom  parallel  robot  previously  developed  by 
Dr.  Cleary.  The  system  will  provide  high  bandwidth,  precision  motion,  and  force  control. 
Some  results  from  this  project  are  shown  in  Figures  1  and  2  and  more  details  can  be 
found  in  [Ding  2008b]. 


Figure  1 :  Parallel  mechanism  robotic  device  holding  ultrasound  probe 


Figure  2:  Experimental  setup  with  tissue  phantom  (ultrasound  image  can  be  vaguely  seen 

on  laptop) 
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3.1.2  Accomplishment  2:  Rotating  Needle  Driver 

The  second  major  accomplishment  was  the  development  of  a  rotating  needle  driver  for 
the  Johns  Hopkins  robotic  system.  The  robotic  system  was  an  updated  version  of  the 
“AcuBof  ’  system  built  by  the  Urology  Robotics  Laboratory  at  Johns  Hopkins  Medical 
Institutions.  The  original  AcuBot  comprises  the  “PAKY”  (Percutaneous  Access  of  the 
KidneY)  needle  driver,  the  “RCM”  (Remote  Center  of  Motion)  orientation  module,  and 
joystick  control.  A  three  degree  of  freedom  Cartesian  stage,  passive  positioning  S-arm, 
and  “bridge  frame”  enable  the  system  to  achieve  a  compact  and  flexible  design  for 
interventions  at  multiple  points  along  the  body. 

Prior  experiences  using  the  AcuBot  system  in  clinical  trials  for  spinal  nerve  blocks 
showed  the  need  for  some  system  enhancements.  The  main  focus  of  these  enhancements 
was  a  complete  redesign  of  the  needle  driver.  Three  new  components  were  added  to  the 
needle  driver:  (1)  a  mechanism  to  spin  the  needle,  (2)  force  sensors,  and  (3)  a  needle 
release  mechanism.  A  picture  is  shown  in  Figure  3  and  details  are  given  in  [Shah  2008], 


Figure  3:  (left)  Rotating  needle  driver  (RND),  b  Remote  center  of  motion  (RCM)  orientation 


module,  c/d  Force  sensor,  e/f  Head  and  Barrel  grippers,  g  Needle  hub 
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3.2  Task  2:  Rehabilitation  Robotics 

There  were  two  major  accomplishments  during  the  last  reporting  period: 

1)  Continued  development  of  a  rehabilitation  arm  exoskeleton  [Carignan  2008a] 

2)  Development  of  a  haptic  control  interface  for  a  motorized  exercise  machine 
[Carignan  2008b] 


3.2.1  Accomplishment  1:  Rehabilitation  Arm  Exoskeleton 

For  the  rehabilitation  arm  exoskeleton  (Figure  4),  a  control  methodology  was  developed 
for  modulating  shoulder  impedance  during  physical  therapy.  Setting  the  remote  center  of 
compliance  at  the  shoulder  allows  the  exoskeleton  to  enact  resistance  training  protocols 
that  strengthen  the  rotator  cuff  and  other  joint  musculature  supporting  the  shoulder 
complex.  The  rotational  kinematics  for  the  shoulder  were  derived,  and  then  the  torques 
applied  at  the  shoulder  were  estimated  using  force  sensors  placed  at  the  hand  and  elbow 
interfaces.  Impedance  and  admittance  control  schemes  were  both  developed  for  realizing 
isolateral  strengthening  exercises,  and  some  preliminary  experimental  results  were 
presented  for  implementation  [Carignan  2008a]. 


Figure  4:  The  MGA  Exoskeleton  has  five  powered  joints  including  a  three-axis 
intersecting  shoulder  and  a  scapula  elevation  joint 
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3.2.2  Accomplishment  2:  Haptic  Control  Interface 

For  the  haptic  interface  (Figure  5),  a  powered  arm  curl  machine  for  modulating  resistance 
“on-the-fly”  during  strength  training  and  rehabilitation  was  modified.  Force  signals  from 
a  load  cell  were  input  to  an  impedance  loop  based  on  the  desired  resistance  law,  which 
then  outputs  position  commands  to  a  servomotor.  The  kinematics  between  the  actuator 
drive  and  arm  curl  angle  were  derived,  and  the  admittance  control  implementation  used 
for  realizing  the  resistance  laws  were  described.  Preliminary  experimental  results  were 
presented  for  viscous  and  inertial  control  laws  [Carignan  2008b], 


Figure  5:  Keiser  Arm  Curl  250  retrofitted  with  UltraMotion  actuator 


During  the  last  reporting  period,  dedicated  funding  for  this  task  was  obtained  under  Army 
grant  W81XWFI-08-2-0010,  so  this  task  will  no  longer  be  reported  under  this  contract  in 
future  reports. 


3.3  Task  3:  Interventional  Suite  of  the  Future 

This  task  was  the  focus  of  much  of  the  work  on  this  grant  during  the  last  reporting  period. 
Here  we  report  on  the  following  accomplishments: 

1)  The  continued  development  of  our  open  source  software  package  the  image- 
guided  surgery  toolkit  IGSTK  [Enquobahrie  2007] 

2)  Demonstration  of  image-guided  vertebroplasty  using  this  software  on  a  phantom 
spine  model  [Ding  2008a] 
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3)  Abdominal  phantom  evaluation  using  this  software  and  a  custom  torso  phantom 
[Lin  2008] 

4)  Evaluation  of  electromagnetic  tracking  in  several  clinical  environments  [Wilson 
2008] 

5)  Development  of  an  image-guided  system  for  treatment  planning  and  navigation 
for  radiofrequency  ablation  of  liver  tumors  [Banovac  2007] 

6)  Performed  studies  to  form  a  rationale  for  PET/CT-guided  interventional 
procedures  and  assess  relative  benefits  [Wong  2007] 

3.3.1  Accomplishment  1:  Open  Source  Software  Package 

IGSTK  is  an  open  source  C++  software  library  that  provides  the  basic  components 
needed  to  develop  image-guided  surgery  applications  [Enquobahrie  2007],  It  is  intended 
for  fast  prototyping  and  development  of  image-guided  surgery  applications.  The  toolkit 
was  developed  through  collaboration  between  academic  and  industry  partners.  Because 
IGSTK  was  designed  for  safety-critical  applications,  the  development  team  has  adopted 
lightweight  software  processes  that  emphasizes  safety  and  robustness  while,  at  the  same 
time,  supporting  geographically  separated  developers.  A  software  process  that  is 
philosophically  similar  to  agile  software  methods  was  adopted  emphasizing  iterative, 
incremental,  and  test  driven  development  principles.  Realizing  that  the  continued  success 
and  viability  of  an  open  source  toolkit  depends  on  a  strong  user  community,  the  IGSTK 
team  is  following  several  key  strategies  to  build  an  active  user  community.  These  include 
maintaining  a  users  and  developers’  mailing  list,  providing  documentation  (application 
programming  interface  reference  document  and  book),  presenting  demonstration 
applications,  and  delivering  tutorial  sessions  at  relevant  scientific  conferences. 
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Figure  6:  IGSTK  component  based  architecture 
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3.3.2  Accomplishment  2:  Image-Guided  Vertebroplasty 

The  IGSTK  software  was  used  to  conduct  phantom  experiments  for  vertebroplasty  spinal 
therapy  based  on  electromagnetic  tracking  of  instruments  in  the  interventional  suite  [Ding 
2008a],  Vertebroplasty  is  a  minimally  invasive  procedure  in  which  bone  cement  is 
pumped  into  a  fractured  vertebral  body  that  has  been  weakened  by  osteoporosis,  long¬ 
term  steroid  use,  or  cancer.  In  this  therapy,  a  trocar  (large  bore  hollow  needle)  is  inserted 
through  the  pedicle  of  the  vertebral  body  which  is  a  narrow  passage  and  requires  great 
skill  on  the  part  of  the  physician  to  avoid  going  outside  of  the  pathway.  In  clinical 
practice,  this  procedure  is  typically  done  using  2D  X-ray  fluoroscopy.  To  investigate  the 
feasibility  of  providing  3D  image  guidance,  we  developed  an  image-guided  system  based 
on  electromagnetic  tracking  and  our  open  source  software  platform  the  Image-Guided 
Surgery  Toolkit  (IGSTK).  The  system  includes  path  planning,  interactive  3D  navigation, 
and  dynamic  referencing.  The  experimental  setup  is  shown  in  Figure  7. 


Figure  7:  In-vitro  experimental  setup  in  the  interventional  radiology  suite 


3.3.3  Accomplishment  3:  Abdominal  Phantom  Evaluation 

Based  on  the  work  described  above,  we  conducted  two  abdominal  phantom  studies 
(Figure  8)  using  our  navigation  system  in  a  clinical  environment  [Lin  2008].  In  the  first 
study,  a  physician  and  medical  resident  performed  a  total  of  18  targeting  passes  in  the 
abdomen  of  an  anthropomorphic  phantom  based  solely  upon  image  guidance.  The 
distance  between  the  target  and  needle  tip  location  was  measured  based  on  confirmatory 


Page  1 1 


Periscopic  Spine  Surgery 


Annual  Report:  21  Dec  06  -  20  Dec  07 


scans  which  gave  an  average  of  3.56  mm.  In  the  second  study,  three  foam  nodules  were 
placed  at  different  depths  in  a  gelatin  phantom.  Ten  targeting  passes  were  attempted  in 
each  of  the  three  depths.  Final  distances  between  the  target  and  needle  tip  were  measured 
which  gave  an  average  of  3.00  mm.  In  addition  to  these  targeting  studies,  we  described 
our  refinement  to  the  standard  four-quadrant  image-guided  navigation  user  interface, 
based  on  clinician  preferences.  We  believe  these  refinements  increase  the  usability  of  our 
system  while  decreasing  targeting  error. 


Figure  8:  Anthropomorphic  phantom  and  experimental  set-up  (phantom  constructed 
by  Dan  Stoianovici,  URobotics  Laboratory,  Johns  Hopkins  University) 


3.3.4  Accomplishment  4:  Electromagnetic  Tracking  Evaluation 

As  part  of  our  research  program,  we  conducted  an  evaluation  of  electromagnetic  tracking 
in  several  clinical  environments  [Wilson  2008],  When  choosing  an  Electromagnetic 
Tracking  System  (EMTS)  for  image-guided  procedures,  it  is  desirable  for  the  system  to 
be  usable  for  different  procedures  and  environments.  Several  factors  influence  this 
choice.  To  date,  the  only  factors  that  have  been  studied  extensively,  are  the  accuracy  and 
the  susceptibility  of  electromagnetic  tracking  systems  to  distortions  caused  by 
ferromagnetic  materials.  In  this  paper  we  provide  a  holistic  overview  of  the  factors  that 
should  be  taken  into  account  when  choosing  an  EMTS.  These  factors  include:  the 
system’s  refresh  rate,  the  number  of  sensors  that  need  to  be  tracked,  the  size  of  the 
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navigated  region,  system  interaction  with  the  environment,  can  the  sensors  be  embedded 
into  the  tools  and  provide  the  desired  transformation  data,  and  tracking  accuracy  and 
robustness.  We  evaluated  the  Aurora  EMTS  (Northern  Digital  Inc.,  Waterloo,  Ontario, 
Canada)  and  the  3D  Guidance  EMTS  with  the  flat-panel  and  the  short-range  field 
generators  (Ascension  Technology  Corp.,  Burlington,  Vermont,  USA)  in  three  clinical 
environments.  We  showed  that  these  systems  are  applicable  to  specific  procedures  or  in 
specific  environments,  but  that,  no  single  system  is  currently  optimal  for  all  environments 
and  procedures  we  evaluated. 


Figure  9:  Electromagnetic  tracking  systems  and  their  components  used  in  this  study 
(left)  Aurora  and  (right)  3D  Guidance 


3.3.5  Accomplishment  5:  RFA  for  Liver  Tumors 

An  image-guided  system  for  optimized  volumetric  treatment  planning  and  execution  for 
radiofrequency  ablation  of  liver  tumors  was  developed  [Banovac  2007],  Radiofrequency 
ablation  of  liver  tumors  is  becoming  an  increasingly  popular  option  for  the  treatment  of 
cancer.  However,  the  procedure  has  several  technical  challenges,  mostly  associated  with 
precision  targeting  of  the  tumor  and  ensuring  complete  ablation  coverage.  In  this  paper 
we  describe  an  image-guided  system  that  we  are  developing  for  improved  visualization 
and  probe  placement  during  these  procedures.  The  system  will  include  a  pre-procedure 
optimization  module  and  an  intra-procedure  guidance  component.  The  system  concept  is 
explained  and  some  preliminary  results  are  given.  While  this  system  is  designed  for 
radiofrequency  ablation  of  liver  tumors,  the  methods  are  applicable  to  other  organs  and 
treatment  methods. 
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Figure  10:  Example  CT  image  from  Georgetown  University  Hospital  showing 
segmented  liver  tumor  (white  on  left  hand  image)  and  overlapping  sphere 
placement  to  cover  tumor  (red  spheres  on  right  hand  image) 


3.3.6  Accomplishment  6:  PET/CT  Guided  Interventional 
Procedures 

Positron-emission  tomography  (PET)  and  PET/CT  (computed  tomography)  are  becoming 
increasingly  important  for  diagnosis  and  treatment  of  cancer.  Clinically  relevant  changes 
can  sometimes  be  seen  on  PET  that  are  not  seen  on  other  imaging  modalities.  However, 
PET  is  not  suitable  for  guiding  biopsy  as  the  images  are  not  obtained  in  real-time. 
Therefore,  our  research  group  has  begun  developing  a  concept  for  PET/CT-guided 
interventional  procedures.  This  paper  presented  the  rationale  for  this  concept,  outlined 
our  research  plan,  and  included  an  initial  study  to  evaluate  the  relative  sensitivity  of  CT 
and  PET/CT  in  detecting  suspicious  lesions  [Wong  2007], 
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Figure  11:  CT  image  (left)  and  PET/CT  fused  image  (right).  With  CT  alone,  the  tumor  in 
the  liver  is  inconspicuous,  but  with  PET/CT  the  tumor  is  easily  seen  as  the  bright  red  spot 
indicated  by  the  crosshairs  (there  is  also  a  second  lesion  in  the  kidney  at  lower  right  on  the 
PET/CT  image).  These  images  clearly  illustrate  the  challenge  of  attempting  image-guided 
biopsy  using  only  CT  or  X-ray  fluoroscopy  for  navigation). 


3.4  Task  4:  Image-Guided  Bronchoscopy 

The  major  accomplishment  during  this  reporting  period  was  the  development  of  an 
electromagnetically  tracked  biopsy  forceps  [Choi  2007],  Transbronchial  biopsy  is  a 
common  method  for  tissue  acquisition  in  diagnosis  of  various  pulmonary  diseases. 
However,  the  procedure  has  technical  limitations  when  performed  on  peripheral  nodules 
due  to  the  small  size  of  peripheral  airways.  To  enhance  the  accuracy  of  the  procedure,  we 
developed  an  electromagnetically  tracked  biopsy  forceps  and  evaluated  the  device  with 
our  image-guided  system.  Our  initial  experimental  results  served  as  a  proof  of  concept  of 
our  system.  Future  work  will  include  additional  phantom  studies  and  animal  experiments. 
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Polyethylene  tube 


Figure  12:  Wireframe  model  and  a  photograph  of  the  EM  tracked  biopsy  forceps 

During  the  last  reporting  period,  dedicated  funding  for  this  task  was  obtained  under  grant 
X81XWH-07-2-0104,  so  this  task  will  no  longer  be  reported  under  this  contract. 

4  Key  Research  Outcomes 

This  section  provides  a  bulleted  list  of  key  research  accomplishments  during  the  entire 
project: 

•  Developed  a  prototype  robotic  system  with  force  feedback  to  assist  the  physician 
in  minimally  invasive  ultrasound  interventions 

•  Completed  a  new  rotating  needle  driver  to  hold  and  spin  the  needle  for  biopsy 
procedures  under  robotic  control 

•  Developed  a  control  methodology  for  the  rehabilitation  arm  exoskeleton  for 
modulating  shoulder  impedance  during  physical  therapy 

•  Added  a  haptic  control  interface  to  powered  arm  curl  machine  for  improved 
capabilities  for  physical  therapy 

•  Demonstrated  image-guided  spinal  vertebroplasty  in  a  phantom  model  using  our 
open  source  software  package  IGSTK 

•  Completed  an  evaluation  study  of  electromagnetic  tracking  in  the  clinical 
enviroment 

•  Developed  an  image-guided  system  for  optimized  volumetric  treatment  planning 
and  execution  for  radiofrequency  ablation  of  liver  tumors 


Page  16 


Periscopic  Spine  Surgery 


Annual  Report:  21  Dec  06  -  20  Dec  07 


•  Investigated  PET/CT  guided  interventional  papers  including  completing  an  initial 
study  to  evaluate  the  relative  sensitivity  of  CT  versus  PET/CT  in  detecting 
suspicious  lesions 

5  Reportable  Outcomes 

This  section  provides  a  list  of  reportable  outcomes.  The  major  product  of  this  year  is  the 
list  of  papers  given  in  Section  7  (References).  Copies  of  these  documents  are  provided  in 
the  appendix. 

In  addition,  several  proposals  to  the  National  Institutes  of  Health  were  submitted  based 
on  this  work.  Several  small  business  grants  were  received  with  company  partners, 
including  one  on  precision  mouse  biopsy  based  on  MR  imaging. 

Our  research  group  also  continued  to  be  prominent  at  major  conferences  in  the  field, 
including  the  SPIE  Medical  Imaging  meeting,  the  Medical  Image  Computing  and 
Computer  Aided  Interventions  (MICCAI)  meeting,  and  the  Computer  Aided  Radiology 
and  Surgery  (CARS)  meeting. 


6  Conclusions 

The  Periscopic  Spine  Surgery  project  has  continued  to  lay  the  ground  work  for 
developing  the  physician  assist  systems  of  the  future.  These  systems  will  incorporate 
robotics,  tracking,  and  visualization  to  improve  the  precision  of  instrument  placement  and 
manipulation  in  minimally  invasive  procedures.  We  have  begun  to  show  the  feasibility  of 
electromagnetic  position  sensing  and  image  overlay  in  the  clinical  environment  and  plan 
to  apply  for  FDA  approval  soon.  We  have  installed  a  new  angiography  suite  at  the 
hospital  incorporating  a  flat  panel  detector  and  many  of  our  future  activities  will  be 
focused  here.  We  have  continued  our  work  on  respiratory  motion  compensation  as  a 
theme  which  cuts  across  many  of  our  efforts.  This  project  has  enabled  the  Georgetown 
team  to  become  a  world  leader  in  the  emerging  fields  of  computer  aided  surgery  and 
medical  robotics.  Our  goal  continues  to  be  to  develop  systems  to  add  the  physician  in 
these  demanding  minimally  invasive  procedures  with  the  ultimate  aim  of  improving 
patient  care. 
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Appendices  (Papers) 


Copies  of  the  1 1  papers  selected  for  this  report  are  reproduced  in  this  section. 

7.1  Banovac  2007:  An  image-guided  system  for ... 

Reprint  begins  on  the  next  page  and  is  two  pages. 
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TUMOR  ABLATION  THERAPIES 


An  image-guided  system  for  optimized  volumetric  treatment 
planning  and  execution  for  radiofrequency  ablation  of  liver  tumors 
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Abstract  Radiofrequency  ablation  of  liver  tumors  is  becoming  an 
increasingly  popular  option  for  the  treatment  of  cancer.  However, 
the  procedure  has  several  technical  challenges,  mostly  associated 
with  precision  targeting  of  the  tumor  and  ensuring  complete 
ablation  coverage.  In  this  paper  we  describe  an  image-guided 
system  that  we  are  developing  for  improved  visualization  and 
probe  placement  during  these  procedures.  The  system  will  include 
a  pre-procedure  optimization  module  and  an  intra-procedure 
guidance  component.  The  system  concept  is  explained  and  some 
preliminary  results  are  given.  While  this  system  is  designed  for 
radiofrequency  ablation  of  liver  tumors,  the  methods  are  appli¬ 
cable  to  other  organs  and  treatment  methods. 

Keywords  Radiofrequency  ablation  •  Liver  tumors  • 

Image  guidance  •  Optimization  •  Planning 

1.  Introduction 

Radiofrequency  ablation  (RFA)  of  liver  tumors  is  a  minimally 
invasive  procedure  that  is  becoming  increasingly  popular  as  a 
treatment  option.  Although  RFA  has  become  a  widely  used 
modality  in  the  primary  treatment  of  hepatocellular  carcinoma 
(HCC)  and  liver  metastases,  the  procedure  has  several  technical 
challenges.  One  of  the  most  significant  challenges  facing  RFA  is 
successfully  targeting  the  tumor  and  providing  adequate  ablation 
coverage  of  the  tumor.  An  improperly  placed  probe  not  only 
provides  incomplete  tumor  ablation,  but  could  also  damage 
nearby  delicate  tissues  or  organs.  Despite  the  availability  of  larger 
electrodes,  many  procedures  require  multiple  ablations  to  main¬ 
tain  a  1  cm  margin.  Multiple  ablations  are  difficult  to  execute  as 
there  is  no  way  to  visualize  the  overlapping  areas  or  to  distinguish 
ablated  areas  from  non-ablated  areas.  Due  to  the  lack  of  any  real¬ 
time  image  guidance,  repeated  insertions  and  positioning  of  the 
electrode  may  be  required  to  hit  the  target  lesion.  Unablated  tissue 
containing  residual  tumor  may  result  in  tumor  recurrence.  Some 
studies  have  shown  that  complete  necrosis  of  the  tumor  was 
achieved  in  only  29-47.6%  of  HCC  lesions  with  diameters  of 
3.1  cm  or  greater  [1,  2]. 

To  meet  these  challenges,  we  are  developing  an  image-guided 
system  based  on  electromagnetic  tracking  for  improved  visuali¬ 
zation  and  probe  placement  during  RFA  of  liver  tumors.  The 
system  will  include  segmentation  of  the  tumor  and  the  liver  vas¬ 
culature  for  improved  visualization  and  path  planning.  The  sys¬ 
tem  will  also  include  a  pre-procedure  module  that  will  allow  for 
volumetric  computer-assisted  3D  treatment  planning.  While  this 
project  is  focused  on  liver  tumors,  the  techniques  developed  will 
be  applicable  to  many  other  diseases  and  organs. 


Pre-treatment  and  post-treatment  CT  scans  of  a  typical  case  are 
shown  in  Fig.  1.  This  3  cm  hepatocellular  carcinoma  is  a  recur¬ 
rence  of  disease  of  a  previously  resected  primary  liver  tumor. 
Repeat  surgery  was  avoided  by  treating  this  tumor  with  percu¬ 
taneous  RFA.  Note  that  there  is  a  clean  margin  of  dark  and 
devascularized,  dead  tissue  including  the  tumor  itself  and  a 
surrounding  1  cm  margin  of  normal  tissue. 

2.  Methods 

The  integrated  system  will  consist  of  two  major  components: 

1.  a  pre-procedure  planning  component  for  liver  and  tumor 
segmentation  and  optimization  of  a  treatment  plan 

2.  an  intra-procedure  component  for  providing  image  guidance 
during  the  procedure 

2.1.  Pre-procedure  planning  and  volumetric  optimization 

We  intend  to  employ  mathematical  optimization  methods  to  de¬ 
velop  a  general  treatment  planning  methodology  as  described 
below.  Our  treatment  planning  approach  will  be  guided  by  these 
five  criteria: 

1.  Minimize  the  number  of  ablations.  Fewer  ablations  mean 
shorter  treatment  times  and  less  chance  for  complications. 

2.  Limit  the  number  of  probe  insertions.  Clinical  considerations 
limit  the  number  of  perforations  of  the  liver  capsule  by  the 
RFA  probe  to  at  most  3  or  4  and  preferably  just  1  or  2.  The 
optimization  model  will  take  into  account  the  number  of  these 
perforations  and  favor  reinsertions  of  the  probe  through  the 
same  entry  point.  The  model  will  include  the  possibility  of 
performing  several  ablations  along  the  same  linear  trajectory, 
as  is  common  in  clinical  practice. 

3.  Probe  trajectory  constraints.  The  model  will  account  for 
physical  constraints  imposed  by  ribs,  vessels,  and  other  organs 
which  may  restrict  possible  trajectories. 

4.  Irregularly  shaped  tumors.  The  optimization  model  will  be 
designed  to  use  segmented  image  data  from  the  patient  and 
will  not  pre-suppose  a  particular  tumor  shape.  This  will  make 
this  treatment  planning  method  much  more  general  and  in¬ 
crease  the  usefulness  of  the  results. 

5.  Controlling  unnecessary  damage  to  healthy  tissue.  The 
burned  volume  from  the  overlapping  probes  will  be  estimated 
in  the  model  with  the  goal  of  keeping  it  as  small  as  possible 
while  fully  covering  the  tumor  and  margin.  This  will  minimize 
damage  to  healthy  tissue  surrounding  the  tumor. 

2.2.  Integrated  system  and  intra-procedure  component 

The  integrated  system  will  include  the  following  components: 

1.  Segmentation  module. 

2.  Treatment  planning  module. 

3.  Commercial  electromagnetic  tracking  system  (AURORA 
from  Northern  Digital  Inc.,  Waterloo,  Canada). 

4.  User  interface  so  that  the  physician  can  view  the  segmentation 
results  (and  edit  them  if  needed);  specify  the  possible  insertion 
points  for  the  trajectory  planning  and  observe/modify  the 
trajectory  planning  results;  and  carry  out  the  intervention. 

Here  we  describe  how  we  envision  using  the  system  in 
clinical  practice.  A  flowchart  of  these  steps  is  shown  in  Fig.  2. 
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Fig.  1  a  Before  treatment  note  the  low  attenuation  liver  mass 
in  the  posterior  segment  of  the  right  hepatic  lobe  abutting  the 
liver  capsule,  b  After  treatment  note  a  clean  margin  of  dark, 
devascularized,  ablated  tissue  including  the  original  tumor  and  the 
surrounding  tissue 


Fig.  3  Example  CT  image  from  Georgetown  University  Hospital 
showing  segmented  liver  tumor  (white  on  left  hand  image)  and 
overlapping  sphere  placement  to  cover  tumor  (red  spheres  on  right 
hand  image) 


Fig.  2  Workflow  for  integrated  system 

The  patient  will  be  prepared  by  placing  three  or  more  fiducials 
above  the  liver.  Contrast  enhanced  CT  images  will  be  obtained 
from  the  lung  base  through  the  abdomen  and  reconstructed 
to  1.0  mm.  The  relevant  anatomy  will  be  segmented,  including 
the  tumor  and  a  1  cm  margin.  Treatment  planning  to  determine 
overlapping  probe  placement  will  be  completed  based  on  the 
constraints  described  in  Sect.  2.1.  The  AURORA  electromag¬ 
netic  tracking  system  will  be  positioned  near  the  patient  and 
enabled.  Image  registration  will  be  performed  using  paired 
point  matching  [3].  The  system  will  then  provide  the  physician 
with  targeting  assistance  and  image  overlay  so  that  the  treat¬ 
ment  plan  can  be  carried  out. 


3.  Results 

To  date,  we  have  some  preliminary  results  for  the  volumetric 
treatment  planning  component.  Testing  was  done  on  several 
synthetic  data  sets  with  known  optimal  solutions  and  on  two  CT 
data  sets  from  actual  liver  hepatomas  (with  tri-axial  diameters  in 
cm  of  3.1  x  2.3  x  5.4  and  3.4  x  3.5  x  3.3,  respectively).  Results 
using  synthetic  data  show  that  solution  quality  is  very  good. 
Computational  times  range  from  seconds  to  less  than  6  minutes  in 
all  cases.  Assuming  a  5  cm  elliptical  array  based  on  the  LaVeen 
RFA  system,  both  hepatoma  cases  were  executed  with  1  cm  sur¬ 
gical  margin.  The  first  case  was  covered  with  six  ablations  while 
the  second  tumor  was  covered  with  four  ablations.  An  example 
result  is  shown  in  Fig.  3.  For  this  example,  a  rough  segmentation 
was  done  manually  to  outline  the  tumor,  followed  by  treatment 
planning  as  shown. 
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Controlling  Shoulder  Impedance  in  a 
Rehabilitation  Arm  Exoskeleton 


Craig  Carignan  and  Michael  Naylor 


Abstract — A  methodology  is  developed  to  enable  compliance 
control  of  the  shoulder  joint  in  an  arm  exoskeleton  developed 
for  physical  therapy.  Setting  the  remote  center  of  compliance 
at  the  shoulder  will  allow  clinicians  to  enact  resistance  train¬ 
ing  protocols  to  strengthen  the  rotator  cuff  and  other  joint 
musculature  supporting  the  shoulder  complex.  The  rotational 
kinematics  for  the  shoulder  are  first  derived,  and  then  the 
torques  applied  at  the  shoulder  are  estimated  using  force 
sensors  placed  at  the  hand  and  elbow  interfaces.  Impedance 
and  admittance  control  schemes  are  both  developed  to  realize 
isolateral  strengthening  exercises,  and  some  preliminary  exper¬ 
imental  results  are  presented  for  implementation  on  an  arm 
exoskeleton  currently  under  development. 

I.  INTRODUCTION 

In  most  manipulator  applications,  the  remote  center  of 
compliance  is  located  at  the  tool  tip  using  force  readings 
from  a  sensor  located  in  the  wrist.  Likewise,  exoskeletons 
developed  for  virtual  reality  (VR)  applications  usually  reflect 
forces  at  the  hand  resulting  from  interaction  with  virtual 
environments  [1].  This  type  of  force  reflection  can  be  met 
by  using  a  central  controller  to  simultaneously  move  all 
of  the  exoskeleton  joints  to  exert  a  desired  force  at  the 
hand.  However,  this  strategy  is  ineffective  for  rehabilitation 
applications  where  individual  arm  joints  are  being  targeted 
for  physical  therapy. 

In  a  rehabilitation  arm  exoskeleton,  the  remote  center  of 
compliance  is  any  joint-muscle  group  in  the  arm  being  tar¬ 
geted  for  therapy.  For  example,  during  the  shoulder  abduction 
exercise  shown  in  Fig.  1,  the  resistance  about  the  shoulder 
abduction  axis  needs  to  be  controlled  by  the  exoskeleton 
over  the  range  of  motion.  An  additional  complication  is  that 
a  force-torque  sensor  placed  at  the  wrist  or  hand  does  not 
alone  provide  enough  information  to  determine  the  torques 
in  the  shoulder  joint.  Therefore  alternative  force  sensor 
emplacement  strategies  also  need  to  be  investigated. 

In  this  article,  dual  impedance- admittance  control  ap¬ 
proaches  are  investigated  for  modulating  impedance  in  the 
shoulder  joint  during  exercise  therapy.  The  shoulder- axis 
can  either  be  fixed  or  vary  with  configuration  of  the  arm. 
Impedance  control  schemes  are  explored  that  use  force- 
torque  sensors  placed  at  the  hand  and  elbow  to  estimate  the 
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Fig.  1.  The  MGA  Exoskeleton  has  five  powered  joints  including  a  three- 
axis  intersecting  shoulder  and  a  scapula  elevation  joint. 


applied  forces.  Some  preliminary  test  results  are  presented 
for  implementing  impedance  control  for  realizing  isolateral 
exercise  protocols. 

II.  PREVIOUS  WORK 

Most  arm  exoskeletons  built  to-date  were  developed  as 
either  force-reflecting  master  arms  for  teleoperation  or  as 
haptic  devices  for  virtual  reality  applications  [1].  In  VR 
applications,  “contact”  forces  are  imparted  at  the  handle  of 
the  exoskeleton  that  replicate  forces  sensed  by  a  slave  arm 
or  by  interaction  with  a  virtual  environment.  A  basic  form 
of  impedance  control  is  usually  implemented  in  which  the 
Cartesian  forces  at  the  handle  are  mapped  into  joint  torque 
commands  using  the  Jacobian  [10].  This  approach  eliminates 
the  need  to  compute  the  inverse  kinematics  and  is  stable  at 
low  impedances. 

The  main  drawback  of  impedance  control  is  that  good 
force  replication  at  the  handle  requires  compensation  of  the 
natural  dynamics  of  the  exoskeleton,  such  as  gravity  loading 
and  drive  friction.  A  force  loop  wrapped  around  the  force 
sensor  can  reduce  unmodeled  effects  [2],  but  it  can  also 
easily  destabilize  the  system.  The  Exoskeleton  Arm-Master 
[1]  and  the  L-Exos  Exo skeleton  [7]  are  classic  examples  of 
exoskeletons  that  use  this  approach. 

An  alternative  approach  called  “admittance”  control  has 
primarily  been  used  to  control  manipulators  used  as  large- 
reach  haptic  devices  [3],  [5].  In  this  approach,  the  sensed 
force  at  the  handle  is  used  as  the  input  to  a  desired  impedance 
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model,  which  outputs  a  desired  motion  to  be  imparted  at  the 
hand.  The  Cartesian  position  is  mapped  into  joint  position 
commands  using  the  inverse  kinematics,  which  are  then  input 
to  a  proportional-derivative  (PD)  servocontroller  to  drive  the 
joints  to  the  desired  position. 

The  main  advantage  of  the  admittance  approach  is  that 
the  high  gains  of  the  joint  position  servo-loop  are  used  to 
reject  unmodeled  dynamics  without  resorting  to  model  feed¬ 
forward.  However,  it  has  the  major  drawback  of  instability 
for  high  admittance  (low  impedance),  which  is  the  opposite 
of  impedance  control  [9].  The  Sensor  Arm  [11]  is  an  example 
of  an  exoskeleton  implementing  this  approach,  and  the  more 
recent  ARMin  Exo skeleton  [12]  appears  to  be  able  to  operate 
in  either  admittance  and  impedance  modes. 

Almost  all  exoskeleton  designs  incorporate  a  six-axis  force 
sensor  at  the  gripper  for  determining  forces  applied  at  the 
hand  [7].  Some  designs  use  force-torque  sensors  mounted  on 
the  links  to  obtain  forces  at  other  locations  along  the  arm. 
The  ARMin  reacts  loads  to  the  distal  end  of  the  forearm  link 
through  force-torque  sensor  attached  to  a  wrist  cuff  [12].  The 
Sensor  Arm  [11]  uses  concentric  rings  connected  by  strain 
gauges  to  determine  forces  applied  by  the  arm,  where  the 
inner  ring  is  secured  to  the  limb  using  an  inflatable  bladder. 
Attempting  to  use  torque  cells  at  the  exoskeleton  joints  to 
derive  torques  in  the  human  joints  is  fraught  with  difficulties 
because  the  exoskeleton  joints  do  not  align  with  the  human 
joints.  Thus  this  approach  is  rarely  used. 


III.  SHOULDER  KINEMATICS 

The  human  shoulder  (glenohumeral)  joint  is  a  ball- 
and-socket  joint  capable  of  abduction/adduction,  flex¬ 
ion/extension,  and  medial/lateral  rotation.  For  a  serial  ma¬ 
nipulator  to  replicate  this  motion,  three  serially  connected 
rotational  joints  with  mutually  intersecting  axes  are  needed. 
In  addition,  the  glenohumeral  joint  translates  along  the 
surface  of  a  sphere  as  the  humerus  (upper  arm)  elevates, 
so  the  ability  to  replicated  scapulo-humeral  rhythm  is  key  to 
realizing  natural  movement  of  the  shoulder. 

Figure  2  shows  the  kinematic  configuration  of  the  MGA 
Exoskeleton  along  with  and  Denavit-Hartenburg  (D-H)  link 
frame  assignments  [6] .  The  D-H  parameters  for  the  kinemat¬ 
ics  are  given  in  Table  I  except  for  the  scapula  joint  0  which 
is  mounted  perpendicular  to  the  torso.  The  angle  between 
the  zo  and  zi  axis  is  30°,  and  the  angle  between  the  zs  and 
^4  axis  is  45°.  The  scapula,  upper  arm,  and  forearm  links  all 
have  passive  sliding  joints  to  accommodate  variable  subject 
geometry:  Ls  =  14.0  —  25.6  cm,  Ljj  =  27.3  —  31.3  cm,  and 
Lp  =  30.0  —  39.0  cm.  The  displacement  of  the  force  sensors 
along  the  z-axis  of  their  respective  frames  are  Lsh  =  6.72 
cm  and  L$e  =  7.62  cm. 

A.  Arbitrary  Shoulder  Rotation 

The  shoulder  rotation  is  defined  as  the  orientation  of  the 
upper  arm  relative  to  the  body  frame  {B}\ 


(1) 


Fig.  2.  CAD  drawing  of  MGA  Exoskeleton  shoulder  showing  link  frame 
assignments. 


TABLE  I 

D-H  PARAMETERS  FOR  THE  MGA  EXOSKELETON. 


link 

CLi-l 

OLi- 1 

di 

0i*  home 

i 

(cm) 

(deg) 

(cm) 

(deg) 

1 

0 

+30 

0 

+90 

2 

0 

-90 

0 

-105 

3 

0 

+90 

a/2  Lv 

-90 

4 

0 

-45 

-Lu 

0 

5 

0 

-90 

Lf 

0 

where  B Rq  =  Rx{~0o)  and  frame  {U}  is  co-located 
with  frame  {3}  but  rotated  45°  about  the  xs  axis,  sRu  = 
Rx{ 45°).  Rx{0)  is  defined  as  the  pure  x-axis  rotation 


Rx{0) 


1  0  0 
0  cos(0)  —sin(0) 
0  sin(Q)  cos(Q ) 


(2) 


°i?3  can  be  found  by  using  the  D-H  Table  to  find  the 
local  link  transformations  lRi+ 1  and  cascading  the  resulting 
rotation  matrices  for  links  1-3.  The  direction  of  the  humeral 
axis  in  the  body  frame  {B}  is  given  by  Bzp,  the  third  column 
of  bRu ,  which  is  used  to  compute  the  axis  of  rotation  for 
internal/external  rotation  exercises. 

The  rotational  velocity  of  the  upper  arm  relative  to  the 
body  frame  {B}  is  found  from 
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B  td  _  Bn  On  3n 

Hu  —  Ho  Ho  Hu 


blju  =  B  JjjOs  (3) 

where  Os  =  [0q  0\  62  Of\T  represents  the  vector  of  shoulder 
joint  angles  0-3  and  B  Jjj  is  3  x  4  rotational  Jacobian  found 
from  the  forward  kinematics. 

B.  Self-Motion  Shoulder  Rotation 

In  some  instances,  the  desired  axis  of  shoulder  rotation 
might  be  along  a  straight  line  from  the  shoulder  to  the  wrist 
as  shown  in  Figure  3.  Because  the  axis  passes  through  the 
wrist,  rotation  about  this  axis  or  “elbow  orbit”  produces  no 
motion  of  the  wrist  and  is  thus  referred  to  as  “self-motion”. 
The  elbow  “orbit”  angle  f  is  defined  as  the  angle  that  the 
plane  formed  by  the  points  S,  E,  and  W  makes  with  the 
reference  plane  defined  by  the  reference  vector,  v,  and  the 
shoulder- wrist  vector,  pw  [8]. 


Fig.  3.  Self-motion  of  the  arm  or  “elbow  orbit”  occurs  about  a  line  from 
the  shoulder  to  the  wrist. 


Let  the  vectors  from  the  shoulder  to  the  wrist  and  elbow 
be  defined  as  pw  and  pe,  respectively,  and  let  v  denote  an 
arbitrary  fixed  unit  reference  vector  in  frame  0.  Now  let  the 
following  auxiliary  vectors  be  defined  as 


Pd 

=  PwiPwPe) 

(4) 

Pp 

^3 

1 

III 

(5) 

Pi 

=  ( Pw  X  v)  X  Pw 

(6) 

The  roll  angle  of  the  SEW  plane  or  “elbow  orbit  angle”  is 
defined  as  the  angle  between  pp  and  pa 


tancf)  = 


pjipe  XPp) 
P£tPp 


(7) 


Using  the  forward  kinematics  to  compute  pw  and  pe,  f 
is  determined  by  performing  the  vector  operation  in  (7) 
numerically  and  then  taking  the  arctangent  of  the  result. 


IV.  SHOULDER  TORQUES 

For  both  monitoring  and  control  purposes,  it  will  be 
important  to  determine  the  torques  being  exerted  by  the 
exoskeleton  about  the  shoulder  joint.  Since  there  is  no  direct 
way  to  measure  these  torques,  force  sensors  located  at  the 
hand  and  elbow  contact  points  can  be  used  to  estimate  the 
torques.  For  general  shoulder  rotations,  the  shoulder  torques 
can  be  decomposed  into  those  perpendicular  to  the  humeral 
axis  and  those  about  the  humeral  axis.  The  perpendicular 
torques  are  determined  using  the  elbow  sensor,  and  torques 
about  the  humerus  axis  are  estimated  using  the  hand  sensor. 

A.  Azimuth/Elevation  Torques 

The  shoulder  azimuth  and  elevation  torques  are  estimated 
by  projecting  the  force  and  moment  from  the  elbow  sensor 
to  frame  {0}.  The  elbow  sensor  is  fixed  to  the  upper  arm 
frame  {U}  at  a  distance  Lse  along  the  elbow  pitch  axis 
£4  and  oriented  at  an  angle  7 se  about  Z4  so  that  uPse  = 
[0  0  Lse]t  and  u RSe  =  Rz(jse)  where 


Rz(0) 


cos(  7)  —sin(  7)  0 

sinfy)  cos(  7)  0 

0  0  1 


(8) 


is  a  pure  rotation  about  the  z-axis.  The  force  /  and  moment 
n  in  frame  {U}  are  calculated  from  the  elbow  sensor  force 
and  moment  readings  fsE  and  use  as  follows  [6] 


U  fu  =  U  Rse/se  (9) 

u njj  =  upse  x  u fu  +  U Rse^ise  (10) 

The  moment  in  frame  {0}  is  then  found  from  (9)  and  (10) 
using 


°n 0  =  °pu  x  °RuU fu  +  °RuUnu  (U) 

which  can  then  be  converted  to  frame  {B}  coordinates  by 
pre-multiplying  (11)  by  bRq. 

B.  Humeral  Torque 

The  torque  about  the  upper  arm  is  found  by  taking  the 
component  of  the  hand  force  that  is  tangent  to  the  humeral 
axis,  i.e.  the  Z4  direction.  The  hand  sensor  is  fixed  to  frame 
{5}  at  a  distance  Lsh  along  the  2:5 -axis  and  oriented  at 
an  angle  7 sh  about  Z5  so  that  5psh  =  [0  0  Lsh]t  and 
5Rsh  =  Rz(lSH )•  The  hand  sensor  force  in  frame  {4} 
coordinates  is  found  from 

4/5  =  ^  Rb*  RsH  f SH  (12) 

The  humeral  torque  can  be  determined  from  the  z-component 
of  the  hand  frame  force  and  the  lever  arm  provided  by  the 
perpendicular  component  of  the  forearm  relative  to  the  upper 
arm 


tua  =  ( LF  +  LSH)sin(d4)4f5  •  z  (13) 

where  64  is  the  elbow  flexion  angle  which  is  fully  extended 
at  0°. 
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C.  Elbow  Orbit  Torque 

The  z-component  of  the  elbow  force  sensor  in  fsE  can 
also  be  used  to  determine  the  torque,  r^,  exerted  about  the 
shoulder- wrist  axis,  pw.  The  elbow  orbit  torque  is  calculated 
by  taking  the  product  of  the  the  z-component  of  the  force 
and  multiplying  it  by  the  moment  arm 

T<f,  =  \Pp\  fsE  •  Z  (14) 

where  pp  is  the  minimum  distance  from  the  elbow  to  SW 
found  using  (5). 

V.  ISO-LATERAL  EXERCISE  PROTOCOLS 

Iso-lateral  exercises  are  those  that  occur  around  a  single 
rotation  axis  of  the  shoulder  and  closely  resemble  those 
performed  manually  with  dumbbells,  rubber  tubing,  and 
exercise  machines  [13].  Examples  of  shoulder  rotation  exer¬ 
cises  include  internal/external  rotation  and  shoulder  abduc¬ 
tion/adduction  as  shown  in  Figures  4  and  5,  respectively. 
In  isotonic  exercises,  the  motion  of  the  shoulder  joints  is 
determined  by  the  resistance  about  the  desired  shoulder  axis 
of  rotation.  In  self-motion  exercises,  the  axis  of  rotation  is 
automatically  specified  by  the  position  of  the  wrist. 


Fig.  4.  Exoskeleton  shown  performing  external/internal  rotation  at  about 
90°  elbow  flexion. 

Some  common  iso-lateral  exercises  are  shown  in  Table  II. 
The  second  column  indicates  the  plane  of  motion,  and  the 
third  column  indicates  the  axis  of  rotation.  The  final  column 
specifies  the  azimuth  and  elevation  of  the  humerus  during 
the  exercise.  Azimuth  corresponds  to  the  rotation  about  the 
longitudinal  axis  zb  (0°  is  straight  ahead)  and  elevation  is 
the  angle  the  humerus  makes  with  the  longitudinal  axis  (0° 
is  straight  down). 

Exercises  are  implemented  using  the  modular  control 
architecture  shown  in  Figure  6  [4].  The  exercise  protocol 
is  first  parsed  into  a  control  mode  based  on  the  desired 
activation  of  arm  joint  variables.  This  code  then  determines 
which  controller  should  be  activated  for  the  possible  com¬ 
binations  of  arm  groups:  scapula,  shoulder,  elbow  pitch, 


Fig.  5.  Exoskeleton  shown  at  90°  shoulder  abduction. 
TABLE  II 

SHOULDER  ISO-LATERAL  EXERCISES. 


Exercise 

Plane  of  Motion 

Rotation  Axis 

Azim./Elev. 

Ab/Adduction 

Flex/Extension 

Ab/Adduction 

Elevation 

Int/Ext  Rotation 

Elbow  Orbit 

frontal 

saggital 

transverse 

scapula 

_Lhumerus 

_L  shoulder- wrist 

[1.0,  Oj 
[0,1,0] 

[0,0, 1] 

[•v/3/2, 1/2, 0] 
SE,  B  zjj 

SW,  °P5 

90°/0°-90° 

0°/0°-90° 

0°-90°/90° 

60°/0°-90° 

-/- 

-/- 

and  elbow  orbit.  These  groups  can  implement  impedance 
and  admittance  modes  depending  upon  the  desired  level  of 
impedance  and  torque  sensing  accuracy.  In  particular,  the 
shoulder  impedance  controller  and  elbow  orbit  admittance 
controller  are  discussed  in  more  detail  below. 


Fig.  6.  Control  modules  are  activated  by  the  mode  selector  depending  upon 
the  exercise  protocol  (AC  =  admittance  control,  IC  =  impedance  control, 
PD  =  proportional-derivative  control). 

A.  Shoulder  Impedance  Module 

The  shoulder  impedance  controller  is  generally  used  for 
low  resistance  shoulder  rotation  exercises,  particularly  shoul¬ 
der  abduction  and  extension.  The  desired  impedance  is  mul¬ 
tiplied  by  the  angular  velocity  of  the  glenhumeral  (GH)  joint 
shown  in  Figure  7  to  produce  a  desired  Cartesian  torque  Tdes . 
The  desired  torque  and  “sensed”  torque  are  then  differenced 
to  form  a  torque  error  and  multiplied  by  a  feedback  gain  Kp. 
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The  desired  torque  and  feedback  error  are  then  converted 
back  to  joint  coordinates  to  produce  a  desired  torque  r^es • 
The  desired  torque  and  feedforward  compensation  Tfwd  are 
then  summed  to  form  the  control  command  to  the  motors. 


IMPEDANCE  CONTROLLER 


Fig.  7.  Impedance  controller  used  for  shoulder  axis  rotation. 

The  desired  stiffness  and  damping  about  the  axis  of 
rotation  are  set  to  the  desired  values  in  ZcHd  while  the 
impedances  about  the  other  two  axes  are  set  to  very  high 
values.  Because  the  desired  axis  of  rotation  does  not  have  to 
be  aligned  with  a  base  frame  axis,  the  z-axis  of  a  rotation 
frame  {C}  is  aligned  with  the  desired  axis  of  rotation,  and 
then  bRc  is  the  transformation  between  the  rotation  frame 
and  the  body  frame.  Thus,  the  desired  stiffness  in  ZcHd  can 
be  found  from 


KGHd  =  bRcKcbRI  (15) 

where  Kc  is  the  stiffness  in  the  rotation  frame.  An  analogous 
procedure  is  used  to  find  the  damping  impedance  BcHd 
where  Kc  is  replaced  by  Be  in  (15).  If  the  rotation  is 
about  the  humeral  axis,  then  the  z-axis  of  the  compliance 
frame  aligns  with  the  humerus  longitudinal  axis  so  that 

bRc  =  bRu ■ 

B.  Elbow  Orbit  Admittance  Module 

The  admittance  controller  shown  in  Figure  8  is  used  to 
drive  the  self-motion  of  the  arm  about  the  pw  axis  while 
being  used  in  conjunction  with  hand  position  commands.  The 
elbow  roll  torque  is  derived  from  the  elbow  force  sensor  as 
derived  in  (14),  which  is  then  multiplied  by  the  desired  elbow 
orbit  impedance  to  produce  a  desired  elbow  orbit  velocity 
4>d-  The  desired  elbow  orbit  velocity  is  then  converted  to  the 
nullspace  component  of  the  desired  joint  velocity  using  the 
inverse  Jacobian,  Jw$.  The  nullspace  velocity  is  then  added 
to  the  particular  solution  of  the  desired  joint  velocity  derived 
from  the  Cartesian  hand  velocity  to  form  the  total  desired 
joint  velocity  Qd-  The  desired  velocity  is  then  integrated  and 
fed  into  a  joint  PD  controller  to  drive  the  exoskeleton. 


IMPEDANCE  COMPENSATOR 


Fig.  8.  The  admittance  controller  utilizes  force  inputs  from  the  elbow  force 
sensor  to  compute  the  commanded  elbow  orbit  rate  based  on  the  desired 
admittance. 

VI.  SHOULDER  EXPERIMENTS 

Several  experiments  were  conducted  to  validate  the  op¬ 
eration  of  the  shoulder  impedance  and  admittance  modules 
during  simulated  exercises.  Since  a  feedforward  model  for 
the  exoskeleton  is  still  under  development,  a  series  of  back- 
drive  tests  were  conducted  for  the  impedance  module.  The 
admittance  module  for  elbow  orbit  was  tested  using  a  pair 
of  load  cells  mounted  on  either  side  of  the  upper  arm  brace 
along  the  elbow  axis.  The  scapula  joint  was  maintained  at 
0°  throughout  these  tests. 

A.  Impedance  Control  Experiments 

The  shoulder  impedance  controller  was  used  to  produce 
an  internal/external  rotation  exercise  by  using  ZcHd  to  select 
the  axis  of  rotation  along  the  humerus.  The  desired  stiffness 
and  damping  about  the  humeral  axis  were  set  to  zero  while 
the  off-axis  values  were  set  to  500  N-m/rad  and  50  N- 
m/rad/sec,  respectively.  No  feedforward  model  is  currently 
available,  so  the  natural  dynamics  of  the  exoskeleton  were 
left  uncompensated.  During  the  test,  the  humerus  was  straight 
down  by  the  side  at  0°  elevation  parallel  to  zb  and  the  elbow 
was  flexed  at  approximately  90°  as  shown  in  Figure  4. 

The  shoulder  was  first  externally  rotated,  then  internally 
rotated  across  the  chest,  and  returned  to  the  start  position. 
The  resulting  angular  velocity  in  base  frame  and  humerus 
torque  are  shown  in  Figure  9.  The  rotation  was  primarily 
restricted  to  the  z-axis  as  expected  since  it  was  parallel  to 
the  humeral  axis.  Since  no  torque  was  being  commanded 
about  the  z-axis  (due  to  the  zero  desired  impedance)  and  the 
humeral  axis  was  parallel  to  the  gravity  vector,  the  sensed 
torque  was  almost  entirely  due  to  the  natural  friction  and 
inertial  loading  in  the  motor/drive  mechanism.  As  seen  in 
Figure  10,  the  rotation  primarily  occurs  in  the  first  shoulder 
joint  which  is  oriented  at  a  30°  angle  with  respect  to  the 
zb-  axis. 

B.  Self-Motion  Experiments 

In  the  next  experiment,  the  subject  executes  a  pure  elbow 
orbit  maneuver  by  “rolling”  the  elbow  about  the  shoulder- 
wrist  line  in  each  direction  at  a  constant  velocity.  The 
desired  impedance  was  set  to  be  a  pure  rotational  damping 
of  Z^des  =  50  N-m/rad/sec  about  the  shoulder- wrist  axis 
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Fig.  9.  Angular  velocity  and  humerus  torque  during  lateral/medial  rotation. 


Fig.  10.  Scapula  and  shoulder  joint  angles  during  shoulder  ab/adduction. 


so  that  the  rotational  velocity  should  be  proportional  to  the 
exerted  torque.  The  resulting  elbow  orbit  angle  and  torque  are 
shown  in  Figure  11.  The  torque  alternates  sign  as  the  subject 
rotates  the  elbow  first  in  one  direction  and  then  the  other. 
The  slope  of  the  elbow  orbit  angle  remains  constant  while 
the  torque  maintains  the  same  value  during  the  sequential 
rotations  indicating  a  pure  damping  impedance. 


Fig.  11.  Elbow  orbit  angle  and  torque  for  b ^  =  50  N-m/rad/s  during 
accommodation  maneuver. 


VII.  CONCLUSION 

Rotational  kinematics  were  developed  for  controlling  the 
shoulder  joints  of  an  arm  exoskeleton  for  several  isolateral 


exercise  protocols.  A  torque  estimation  scheme  based  on  dual 
force  sensors  was  also  developed  for  calculating  the  torques 
applied  on  the  glenohumeral  joint  by  the  exoskeleton.  The 
impedance  control  tests  revealed  backdrive  friction  levels  of 
approximately  40  N-m/rad/sec  that  will  need  to  be  compen¬ 
sated  in  the  final  design.  The  admittance  controller  demon¬ 
strated  good  ability  to  track  a  pure  damping  impedance  for 
elbow  orbit,  which  does  not  rely  on  model  feedforward. 

The  elbow  load  cells  are  currently  being  replaced  by  a 
six-axis  force-torque  sensor  to  enable  full  determination  of 
forces  applied  at  the  elbow  for  shoulder  torque  estimation. 
The  full  impedance  parameters  are  being  determined  for  a 
variety  of  exercise  protocols  that  take  into  account  the  human 
strength  potentials  over  the  range  of  motion.  Clinical  trials 
will  be  conducted  and  compared  with  results  from  manual 
therapy  after  the  control  design  is  complete. 

ACKNOWLEDGMENT 

Thanks  go  to  Mike  Liszka,  John  Tang,  Stephen  Roderick, 
Emmanuel  Wilson,  and  Mike  Perna  who  made  the  MGA 
Exoskeleton  possible.  This  project  was  sponsored  by  the  U.S. 
Army  Telemedicine  and  Advanced  Technology  Research 
Center  (TATRC)  under  Grant  #W81XWH-04-l-0078. 

References 

[1]  G.  C.  Burdea,  Force  And  Touch  Feedback  For  Virtual  Reality.  New 
York:  John  Wiley  and  Sons,  1996. 

[2]  C.  Carignan  and  K.  Cleary,  “Closed-loop  force  control  for  haptic 
simulation  of  virtual  environments,”  The  Electronic  Journal  of  Haptics 
Research  (http://www.haptics-e.org),  vol.  2,  no.  2,  pp.  1-14,  Feb.  2000. 

[3]  C.  Carignan  and  D.  Akin,  “Achieving  impedance  objectives  in  robot 
teleoperation,”  in  Proc.  of  the  IEEE  Conference  on  Robotics  and 
Automation,  1997,  pp.  3487-3492. 

[4]  C.  Carignan,  J.  Tang,  S.  Roderick,  and  M.  Naylor,  “A  configuration- 
space  approach  to  controlling  a  rehabilitation  arm  exoskeleton,”  in  Int. 
Conf.  on  Rehabilitation  Robotics  (ICORR),  Noordwijk,  Netherlands, 
June  2007,  pp.  179-187. 

[5]  C.  Clover,  G.  Luecke,  J.  Troy,  and  W.  McNeely,  “Dynamic  simulation 
of  virtual  mechanisms  with  haptic  feedback  using  industrial  robotics 
equipment,”  in  Proc.  of  the  IEEE  Conference  on  Robotics  and  Au¬ 
tomation,  Apr.  1997,  pp.  724-730. 

[6]  J.  Craig,  Introduction  to  Robotics:  Mechanics  and  Control,  2nd  ed. 
Reading,  Mass.:  Addison- Wesley,  1989. 

[7]  A.  Frisoli,  F.  Rocchi,  S.  Marcheschi,  A.  Dettori,  F.  Salsedo,  and 
M.  Bergamasco,  “A  new  force-feedback  arm  exoskeleton  for  haptic 
interaction  in  virtual  environments,”  in  Proc.  of  the  First  Joint  Euro¬ 
haptics  Conference  and  Symposium  on  Haptic  Interfaces  for  Virtual 
Environment  and  Teleoperator  Systems,  2005,  pp.  195-201. 

[8]  K.  Kreutz-Delgado,  M.  Long,  and  H.  Seraji,  “Kinematic  analysis  of  7 
DOF  manipulators,”  Int.  Journal  of  Robotics  Research,  vol.  11,  no.  5, 
pp.  469-481,  1992. 

[9]  D.  Lawrence,  “Impedance  control  stability  properties  in  common  im¬ 
plementations,”  in  Proc.  IEEE  Int.  Conf.  on  Robotics  and  Automation, 
1988,  pp.  1185-1190. 

[10]  T.  Massie  and  J.  K.  Salisbury,  “The  PHANTOM  haptic  interface:  A 
device  for  probing  virtual  objects,”  in  Proc.  ASME  Winter  Annual 
Meeting:  Symposium  on  Haptic  Interfaces  for  Virtual  Environment  and 
Teleoperator  Systems,  vol.  DSC  55-1,  Nov.  1994,  pp.  295-300. 

[11]  A.  Nakai,  Y.  Kunii,  H.  Hashimoto,  and  F.  Harashima,  “Arm  type  haptic 
human  interface:  Sensor  arm,”  in  Inti.  Conf.  on  Artificial  Reality  and 
Tele-Existence  (ICAT),  Tokyo,  Japan,  Dec.  1997,  pp.  77-84. 

[12]  T.  Nef,  M.  Mihelj,  G.  Colombo,  and  R.  Riener,  “ARMin  -  robot  for 
rehabilitation  of  the  upper  extremities,”  in  Proc.  IEEE  Int.  Conf.  on 
Robotics  and  Automation,  orlando,  2006,  pp.  3152-3157. 

[13]  L.  Peterson  and  P.  Renstrm,  Sports  Injuries  Their  Prevention  and 
Treatment,  3rd  ed.  Champaign,  II.:  Human  Kinetics,  2001. 


Page  29 


Periscopic  Spine  Surgery 


Annual  Report:  21  Dec  06  -  20  Dec  07 


7.3  Carignan  2008b:  A  haptic  control  interface  ... 

Reprint  begins  on  the  next  page  and  is  six  pages. 


Page  30 


2008  IEEE  International  Conference  on 
Robotics  and  Automation 
Pasadena,  CA,  USA,  May  19-23,  2008 


A  Haptic  Control  Interface  for  a 
Motorized  Exercise  Machine 


Craig  R.  Carignan  and  Jonathan  Tang 


Abstract — A  haptic  interface  is  generated  for  a  powered 
arm  curl  machine  for  modulating  resistance  “on-the-fly”  during 
strength  training  and  rehabilitation.  Force  signals  from  a  load 
cell  are  input  to  an  impedance  loop  based  on  the  desired 
resistance  law,  which  then  outputs  position  commands  to  a 
servomotor.  The  kinematics  between  the  actuator  drive  and 
arm  curl  angle  are  derived,  and  the  admittance  control  imple¬ 
mentation  used  for  realizing  the  resistance  laws  is  described. 
Preliminary  experimental  results  are  presented  for  viscous  and 
inertial  control  laws. 

I.  INTRODUCTION 

Resistance  training  has  long  been  recognized  as  key  to 
increasing  muscular  strength  and  endurance  for  both  athletic 
performance  and  neuromuscular  rehabilitation.  The  greatest 
increase  in  strength  is  attained  through  high  resistances  for 
short  durations,  while  low  resistances  for  long  durations  tend 
to  produce  high  muscular  endurance  [1].  Until  the  1970s, 
free  weights  (e.g.  barbells  and  dumbells)  and  Universal 
Gyms  utilizing  weight  stacks  driven  by  cable-pulleys  were 
the  primary  means  of  resistance  training.  During  the  fitness 
craze  in  the  1980s,  Nautilus,  Cybex,  and  other  companies 
introduced  exercise  machines  into  fitness  centers  for  higher 
volume  training.  The  cadre  of  resistance  devices  commer¬ 
cially  available  now  includes  pulley- weight  machines,  spring 
reaction  devices  (spring  or  pneumatic  pistons),  frictional 
devices  (fans  or  brakes),  and  free  weights. 

Many  of  these  machines  offer  some  variability  in  resis¬ 
tance  over  the  range  of  motion  to  account  for  the  strength 
potential  of  the  arm  or  leg  in  different  configurations.  This 
adjustability  is  usually  attained  through  a  series  of  cams  and 
levers  that  make  it  harder  to  push  (or  pull)  the  “weight”  in 
positions  of  greater  biomechanical  advantage.  Although  these 
devices  may  offer  a  level  of  adjustability,  several  researchers 
have  experimentally  demonstrated  that  the  machine  torque 
profiles  do  not  match  the  human  strength  curve  [4].  In 
addition,  machines  also  have  other  issues: 

•  Inertial  resistance  is  not  optimal  because  weights  can 
be  “thrown”  which  causes  uneven  force  over  the  range 
of  motion. 

•  Maximum  resistance  should  be  applied  at  every  position 
in  order  to  increase  strength  throughout  the  range. 

•  Exercises  should  closely  resemble  the  motion  being 
trained  or  rehabilitated,  to  increase  strength. 
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•  Concentric  and  eccentric  motion  are  both  effective  at 
increasing  strength. 

•  External  movement  of  a  disabled  limb  is  still  useful  in 
restoring  its  function. 

An  actively-controlled  exercise  machine  can  help  resolve 
many  of  these  issues.  Different  types  of  resistances  could  be 
generated  and  varied  over  the  range  of  motion.  Bidirectional 
forces  could  be  applied  to  realize  eccentric  as  well  as 
concentric  exercise.  A  powered  machine  could  be  used  as  an 
assistive  device  in  the  beginning  stages  of  physical  therapy 
and  then  gradually  change  over  to  a  resistive  training  device 
as  strength  increases.  In  addition,  strength  training  could  be 
tailored  to  individual  strength  curves,  which  could  slowly  be 
restored  to  normal  following  injury  or  surgery  [13]. 

In  this  work,  a  commercial  curl  machine  was  motorized 
in  order  to  actively  control  the  resistance  during  elbow 
flexion  exercises  (see  Fig.  1).  Active  control  allows  new 
resistance  laws  to  be  implemented  that  have  noninertial 
and  eccentric  characteristics  and  may  be  hard  to  generate 
physically.  Resistance  can  also  be  automatically  adjusted 
over  the  range  of  motion  and  tuned  to  individual  strength 
characteristics.  In  addition,  data  from  onboard  sensors  can 
be  used  to  produce  performance  measures  and  track  patient 
progress. 


Fig.  1.  Keiser  Arm  Curl  250  retrofitted  with  UltraMotion  actuator. 


This  article  begins  with  a  brief  review  in  Section  II  of 
previous  work  in  actively-controlled  exercise  machines.  The 
actuator  and  drive  system  used  in  this  study  are  described 
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in  Section  III  and  a  kinematic  model  is  developed.  The 
hardware  and  control  system  architecture  are  outlined  in 
Section  IV.  Preliminary  experimental  results  for  inertial  and 
damping  resistances  are  given  in  Section  V.  Conclusions 
on  the  implementation  of  the  approach  along  with  some 
directions  for  future  research  are  given  in  Section  VI. 

II.  PREVIOUS  WORK 

The  idea  of  allowing  a  user  to  dynamically  “tune”  machine 
resistance  is  not  new.  Keiser,  Inc.  has  developed  a  product 
line  that  uses  pneumatic  pistons  instead  of  weights  to  provide 
resistance.  The  exerciser  pushes  buttons  on  the  handle  to 
either  increase  or  decrease  air  pressure  in  the  piston,  causing 
a  corresponding  increase  or  decrease  in  resistance.  However, 
the  subject  primarily  uses  this  feature  as  an  alternative  to 
moving  a  pushpin  on  a  weight  stack  and  rarely  changes 
resistance  over  the  course  of  a  repetition. 

A  motorized  version  of  this  concept  is  the  leg  extension 
machine  developed  at  the  University  of  Connecticut  [7]. 
An  electric  motor  that  drives  two  sprockets  is  used  by  the 
machine  to  engage  a  cam  that  increases  or  decreases  the 
angle  of  motion  of  a  weighted  lever  arm.  This  innovation 
allows  the  user  to  adjust  the  resistance  at  any  instant  during  a 
movement  by  operating  a  switch  mounted  on  a  grasp  handle. 
Although  adjustable,  the  resistance  is  still  inertial,  and  the 
device  relies  on  the  human  to  control  the  amount  of  weight. 

Automatic  control  of  exercise  machines  has  also  been 
explored.  A  microprocessor-controlled  leg  press  machine  was 
developed  at  Georgia  Tech  in  the  1980s  [2].  The  device  used 
hydraulic  pistons  that  were  able  to  exert  up  to  300  N  of 
force  and  drive  the  lift  bar  in  both  vertical  and  horizontal 
directions.  Thus,  the  machine  was  capable  of  not  only  tuning 
the  force,  but  also  allowing  paths  in  a  plane  rather  than  along 
a  fixed  arc  or  straight  line. 

Advanced  concepts  for  “smart”  exercise  machines  were 
under  development  at  UC  Berkeley  in  the  mid  1990s  and 
tested  on  a  crank  apparatus  [10].  Investigators  implemented 
a  controller  in  the  form  of  a  nonlinear  dynamic  damper 
that  would  interact  passively  (stably)  with  the  user.  The 
controller  made  use  of  a  force-velocity  biomechanical  model 
to  try  to  extract  maximum  power  during  an  exercise  and 
thus  elicit  more  efficient  training.  These  concepts  were  later 
implemented  into  tunable  brakes  for  a  Nordic  Track  machine 
and  a  stair  stepper  exercise  machine  but  did  not  advance  to 
the  commercial  development  stage  [14]. 

III.  DRIVE  SYSTEM  MODELING 

In  this  study,  an  arm  curl  machine  was  chosen  for  the 
prototype  exercise  device.  In  addition  to  being  a  common 
upper  body  exercise,  elbow  flexion  also  involves  only  three 
major  muscle  groups:  biceps  brachii,  brachialis,  and  brachio- 
radialis  [9] .  Isolation  of  muscle  groups  will  facilitate  clinical 
evaluation  of  the  machine.  To  accelerate  the  development 
process,  a  commercial  exercise  machine  was  retrofitted  with 
an  active  drive  system  rather  than  building  a  new  device  from 
scratch.  This  strategy  allowed  the  development  to  focus  on 
controls  and  software  rather  than  the  hardware. 


The  commercial  version  of  the  Keiser  Arm  Curl  250 
machine  using  a  pneumatic  piston  to  push  on  a  lever  which 
then  pulls  on  the  arm  curl  bar  using  a  rubber  belt  attached  at 
the  other  end  of  the  lever  as  shown  in  Figure  2.  To  achieve 
the  high  control  bandwidths  required  for  this  application,  the 
piston  assembly  was  replaced  with  an  UltraMotion  actuator, 
which  consisted  of  a  lead  screw  with  0.125  in  (0.3175  cm) 
pitch  driven  by  an  electric  motor.  The  lead  screw  had  a  20 
cm  stroke  (greater  than  the  original  piston)  and  was  capable 
of  moving  approximately  45  cm/sec  unloaded,  and  20  cm/sec 
with  a  445  N  load. 


Fig.  2.  View  of  arm  curl  machine  showing  the  linear  actuator  connected 
to  lever  and  belt  drive. 

Because  of  the  complexity  of  the  cam,  lever,  and  drive 
belt  assembly  and  absence  of  mechanical  drawings,  the 
mapping  between  the  lead  screw  displacement  and  the  elbow 
flexion  angle  was  determined  experimentally.  The  motor  was 
commanded  to  move  in  increments  of  5000  encoder  counts, 
and  then  the  angle  of  the  curl  bar  was  read  off  a  protractor 
suspended  from  the  bar  as  shown  in  Figure  3.  The  results 
are  plotted  in  Figure  4. 

Since  the  relationship  is  nonlinear,  a  third  order  polyno¬ 
mial  curve  was  fit  to  the  data  (R=0. 99997)  resulting  in  the 
following  approximation  between  the  elbow  flexion  angle  <f> 
(rad)  and  the  lead  screw  displacement  x  (m) 

4>{x)  =  00  +  0.3337  +  17.275a;  -  58.407a;2  +  506.07a;3  (1) 

where  </> o  is  the  constant  offset  between  the  curl  bar  angle 
and  arm  flexion  angle.  The  Jacobian,  defined  as  J  =  d<p/dx , 
relates  the  elbow  flexion  velocity  to  the  lead  screw  velocity 
and  can  be  found  by  differentiating  (1)  with  respect  to  x 
resulting  in 

J(x)  =  17.275  -  116.81a;  +  1518.2a;2  +  0.0010069a;3  (2) 
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Fig.  3.  Bicep  curl  repetition  starting  from  near  full  extension  through 
maximum  flexion  at  about  110°.  Protractor  on  arm  curl  bar  was  used  to 
generate  Figure  4. 


The  plot  of  the  Jacobian  versus  the  piston  displacement  is 
shown  in  Figure  5.  As  can  be  seen  from  the  graph,  the 
lead  screw  has  minimum  effect  on  the  flexion  angle  at 
approximately  4  cm.  By  invoking  the  same  duality  found 
in  manipulator  kinematics  [6],  the  Jacobian  also  relates  the 
lead  screw  force  F  to  the  flexion  torque  r 


Fig.  4.  Angle  of  curl  bar  versus  the  lead  screw  displacement. 
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0  =  J(x)x  (3) 

F  =  J{x)r  (4) 

Thus  (4)  and  (2)  can  be  used  to  determine  the  torque 
produced  by  a  given  force  at  the  lead  screw.  The  graph 
in  Fig.  6  shows  the  torque  produced  by  a  500  N  force  at 
the  piston  versus  the  maximum  isokinetic  strength  profile 
for  elbow  flexion  [3]  scaled  to  the  same  maximum  peak 
value.  (Note  that  since  the  original  pneumatic  piston  on 
the  Keiser  machine  produces  constant  force,  the  former  is 
also  the  natural  strength  profile  of  the  machine.)  The  plots 
illustrate  the  large  discrepancy  in  elbow  angle  at  which  the 
maximum  torque  occurs.  However,  as  stated  in  the  introduc¬ 
tion,  maximum  resistance  should  be  applied  at  every  position 
throughout  the  range  of  motion  in  order  to  increase  strength, 
i.e.,  the  human  strength  profile  is  optimum.  The  next  section 
will  describe  the  implementation  of  haptic  interfaces  for 
realizing  common  resistance  laws  at  the  elbow  flexor  joint. 

IV.  HAPTIC  CONTROL  INTERFACE 

An  UltraMotion  lead  screw  actuator  with  an  Animatics 
1720  DC  motor  was  used  in  place  of  the  pneumatic  piston 
in  the  original  machine  (see  Fig.  7).  The  motor  drives  the 
lead  screw  via  a  rubber  pulley  with  gear  ratio  1 : 1  attached  to 
the  shaft.  The  angle  of  the  motor  is  determined  using  a  2000 
line  incremental  encoder  mounted  on  the  motor  shaft.  The 
encoder  is  read  using  the  SmartMotor  microprocessor  which 
runs  a  servoloop  at  2000  Hz  based  position  commands  sent 
over  an  RS232  line  from  the  PC. 


Fig.  5.  Jacobian  relating  elbow  flexion  velocity  to  lead  screw  velocity. 

A  Sensotec  Model  31  tension/compression  load  cell  with 
250  lb  (1100  N)  capacity  was  used  to  measure  the  force 
at  the  output  of  the  actuator  and  was  mounted  between  the 
output  of  the  lead  screw  and  the  coupling  to  the  lever  arm 
on  the  machine  (bottom  left  of  Fig.  7).  The  load  cell  voltage 
and  encoder  were  read  directly  by  a  National  Instruments 
PCI-6013  16-bit  data  acquisition  card  in  the  PC  rack  and 
processed  to  produce  position  commands  to  be  sent  back  to 
the  SmartMotor  by  binary  data  transfer  over  the  RS232  line. 

The  dual-loop  admittance  controller  shown  in  Figure  8 
was  implemented  using  a  PC  and  onboard  microprocessor 
to  control  the  drive  assembly  [12],  [5].  This  approach  uses 
an  outer  loop  wrapped  around  a  force  sensor  to  obtain 
the  desired  compliance,  while  a  proportional-derivative  (PD) 
servocontroller  is  used  to  drive  the  lead  screw  to  the  target 
position.  Although  model  feedforward  would  help  reduce  the 
effect  of  friction  and  inertia  in  the  drive  system,  the  high  PD 
gains  in  the  servo  controller  reject  most  of  these  disturbances. 

The  impedance  (outer)  loop  operates  by  using  the  force 
signal  from  the  load  cell  Fs  scaled  by  the  desired  admittance 
to  produce  a  desired  velocity  ±d  command  for  the  lead  screw. 
The  desired  velocity  is  then  integrated  and  multiplied  by  the 
gear  ratio  between  the  motor  and  lead  screw  drive  rj  (0.3175 
cm/rev)  to  obtain  the  desired  angular  position  of  the  motor 
6d.  The  commanded  motor  position  is  then  sent  from  the 
PC  over  the  serial  line  to  the  SmartMotor  servocontroller  to 
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F(t)  =  bdv(t ) 
Z{s)  =  bd 


(6) 

(7) 


where  bd  is  the  desired  viscous  damping  coefficient.  For 
the  cable-pulley- weight  system  shown  in  Figure  10,  the 
resistance  law  is  given  by 


F(t)  =  mdv(t)  -  fw  (8) 

Z(s)  =  mds  (9) 


Fig.  6.  Elbow  flexion  torque  versus  angle  for  constant  force  of  500 
N  applied  at  the  lead  screw.  The  human  strength  profile  is  shown  for 
comparison.  The  force  at  the  hand  can  be  found  by  dividing  the  torque 
by  the  moment  arm  of  0.35  m. 


where  rrid  is  the  desired  mass,  g  is  the  gravitational  accel¬ 
eration,  and  fw  =  nridg .  (Note  that  the  constant  weight  of 
the  mass  is  not  included  in  the  impedance.)  By  contrast,  the 
impedance  for  a  pure  spring  would  be  the  desired  stiffness 
times  the  integral  operator,  Z(s)  =  kd/s. 


Fig.  9.  Damping  or  “dashpot”  resistance  profile. 


Fig.  7.  Keiser  pneumatic  piston  (top)  versus  Ultramotion  lead  screw 
actuator  driven  by  SmartMotor  (bottom). 


drive  the  lead  screw  to  the  desired  position. 

The  resistance  law  is  determined  by  the  desired  relation¬ 
ship  between  the  applied  force  F  and  the  velocity  v  at  the 
handle.  In  general  terms,  this  resistance  is  known  as  the 
mechanical  “impedance”  Z(s)  [8]  and  is  represented  in  the 
frequency  domain  as: 


Fig.  10.  Cable- weight  resistance  profile. 


F(s)  =  Z(s)V(s)  (5) 


SmnRTITlOTOR  SfRUO  COflTROLLCR 


Fig.  8.  Admittance  control  block  diagram. 

For  the  pure  damper  shown  in  Figure  9,  the  resistance  law 
is  given  by 


These  fundamental  impedance  types  can  also  be  superposed 
to  form  more  complex  resistance  laws  such  as  the  mass- 
spring-dashpot 

Z(s)  =  mds  +  bd  +  kd/s  (10) 

The  next  section  presents  some  experimental  results  for 
inertial  and  damping  impedances. 

V.  ADMITTANCE  CONTROL  EXPERIMENTS 

Two  sets  of  preliminary  experiments  were  conducted  to 
validate  operation  of  the  exercise  machine.  In  the  first  set, 
the  admittance  was  set  to  be  a  pure  resistive  damper  with 
an  added  bias  force.  In  the  second  set,  the  admittance  at  the 
linear  actuator  was  set  to  simulate  a  weight  stack  driven  by 
a  cable-pulley  system.  The  sample  rate  of  the  PC  was  150 
Hz,  and  the  SmartMotor  sample  rate  was  2  KHz.  A  low  pass 
filter  with  a  bandwidth  of  10  Hz  was  used  to  filter  out  force 
sensor  noise  and  encoder  noise  due  to  backlash  in  the  system. 
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A.  Viscous  Damping 

In  these  experiments,  the  desired  impedance  was  a  pure 
damper  bd  =  2500  N/m/s.  A  bias  force  of  200  N  was  also 
applied  that  exponentially  rises  to  its  final  value  with  a  decay 
constant  of  0.5  cm.  Thus,  the  desired  force  was 


Fdes  =  — 2500A  -  200(1  -  e-200(*-*o))  (il) 

The  elbow  flexion  angle  and  lead  screw  velocity  versus  time 
are  shown  in  Fig.  11.  The  flexion  angle  peaks  at  about  100°, 
and  the  velocity  of  the  piston  reaches  a  top  speed  of  about 
7  cm/s  which  is  well  within  even  its  fully  loaded  capability. 


CD_ 

O 

o 

-< 


Fig.  11.  Arm  flexion  angle  and  lead  screw  velocity  during  two  repetitions 
with  the  damping  controller. 


The  lead  screw  force  versus  time  for  two  repetitions  is 
shown  in  Fig.  12  alongside  the  velocity  for  reference.  The 
bias  force  rises  rapidly  to  200  N,  with  the  remainder  of  the 
applied  force  due  to  the  damping  impedance.  The  viscous 
damping  peaks  at  about  150  N  when  the  velocity  reaches 
about  6  cm/s,  which  is  consistent  with  a  set  damping  of  2500 
N/m/s.  After  the  elbow  begins  extension  (eccentric  motion), 
the  viscous  force  changes  to  the  same  direction  as  the  bias 
force  causing  a  large  drop  off  in  the  force  curve.  As  seen  in 
the  plot,  the  lead  screw  force  tracks  the  desired  force  in  (11) 
very  closely  throughout  the  motion. 
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Fig.  12.  Lead  screw  force  and  velocity  versus  time  for  the  damping  control 
law. 

The  resulting  torque  about  the  elbow  versus  the  flexion 
angle  for  the  two  cycles  is  shown  in  Fig.  13.  The  top  curve 


is  the  concentric  (flexion)  phase,  and  the  bottom  curve  is 
the  eccentric  (extension)  phase.  The  concentric  force  peaks 
at  approximately  70°  which  is  midway  between  the  human 
and  machine  peaks  shown  previously  in  Fig.  6. 


Flexion  ^ngle(deg) 


Fig.  13.  Flexion  torque  versus  angle  for  concentric  and  eccentric  phases 
of  the  damping  control  experiment. 


B.  Cable-Pulley -Weight  Simulation 

In  these  experiments,  the  machine  was  used  to  simulate 
an  arm  curl  resistance  provided  by  a  mass  on  a  cable-pulley 
system.  Due  to  the  sample  rate  limitations  of  the  RS232  port, 
the  controller  was  not  stable  for  simulated  masses  below 
about  200  kg.  Even  a  6:1  reduction  in  force  between  the 
force  applied  at  the  lead  screw  and  handle  (moment  arm  = 
0.355  m)  would  still  result  in  too  large  of  a  weight  to  pull 
at  the  handle.  Thus,  the  inertial  mass,  m^,  was  set  equal  to 
500  kg  in  (9),  and  the  bias  force  fw  was  set  equal  to  500  N, 
which  is  approximately  equivalent  to  a  50  kg  weight.  Thus, 
the  desired  force  was 


Fdes  =  — 500x  -  250  (12) 

The  resulting  elbow  flexion  angle  and  lead  screw  force  versus 
time  for  a  pair  of  arm  curl  repetitions  is  shown  in  Figure  14. 
The  force  averages  about  250  N  (the  simulated  weight),  and 
the  oscillations  are  characteristic  of  the  undesirable  accel¬ 
eration  and  deceleration  of  the  mass  during  elbow  flexion 
and  extension  seen  on  many  cable-pulley  machines.  From 
a  subjective  point  of  view,  the  controller  provided  a  very 
realistic  simulation  of  a  cable-pulley  machine  although  it  was 
impossible  to  validate  that  forces  were  tracked  correctly  in 
the  absence  of  acceleration  measurements. 

The  resulting  elbow  torque  as  a  function  of  the  flexion 
angle  is  shown  in  Fig.  15.  The  concentric  torques  at  the  start 
of  the  motion  are  larger  than  in  the  eccentric  phase  due  to 
the  forces  needed  to  accelerate  the  mass  from  rest  but  are 
correspondingly  lower  near  the  peak  when  the  mass  reverses 
direction.  The  eccentric  torques  are  higher  on  average  in  the 
cruise  phase  because  of  the  inertial  deceleration  forces.  The 
torque  peaks  at  approximately  70°  which  is  significantly  less 
than  the  isokinetic  human  strength  profile  superposed  on  the 
same  graph. 
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Fig.  14.  Flexion  angle  and  lead  screw  force  versus  time  for  cable  machine 
test. 
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Fig.  15.  Flexion  torque  versus  angle  for  cable  machine  test. 

VI.  CONCLUSIONS 

A  powered  exercise  machine  was  developed  by  retrofitting 
a  commercial  arm  curl  machine  with  a  motorized  lead  screw 
drive.  An  optical  encoder  and  load  cell  provided  sensory 
feedback  to  a  dual-loop  admittance  controller  implemented 
on  a  PC  and  microprocessor.  A  10  Hz  low-pass  filter  was 
used  to  suppress  electrical  noise  introduced  into  the  system 
through  the  force  sensor.  Sample  rates  of  the  compliance 
loop  were  limited  to  150  Hz  by  the  RS232  communications 
link  with  the  SmartMotor,  which  places  an  upper  bound  on 
the  admittance  gains  that  could  be  simulated.  Fortunately, 
resistance  training  typically  requires  high  impedance,  which 
drives  admittance  toward  the  low  end  of  the  spectrum. 

Experiments  conducted  with  constant  damping  and  force 
bias  produced  a  smooth  controlled  motion,  comparable  to 
that  of  the  original  machine.  The  forces  produced  by  the 
admittance  controller  accurately  tracked  the  desired  forces 
indicating  that  the  desired  damping  was  achieved.  The  peak 
torque  occurs  at  a  larger  flexion  angle  than  that  of  a  constant 
force  piston,  but  the  peak  still  falls  short  of  the  isometric 
strength  peak  near  90° .  This  is  because  of  the  zero  boundary 
condition  of  the  damping  resistance  at  maximum  flexion  and 
extension  when  the  velocity  goes  to  zero. 

The  machine  was  also  used  to  simulate  inertial  resistance 
training  using  a  cable-pulley-weight  system.  The  experiment 
produced  a  very  realistic  haptic  interface  to  the  user,  even 
faithfully  replicating  the  “throwing”  motion  of  the  weight 


stack  on  a  cable  often  cited  as  a  significant  disadvantage  of 
inertial  training.  Although  our  goal  was  to  simply  emulate 
an  inertial  trainer,  the  oscillations  could  be  reduced  or 
even  eliminated  by  adding  some  damping  to  the  desired 
impedance. 

Our  goal  was  to  show  that  a  motorized  exercise  ma¬ 
chine  could  be  used  to  achieve  a  wide  variety  of  resis¬ 
tance  laws,  and  this  was  demonstrated  using  damping  and 
inertial  resistance  laws  to  simulated  classes  of  well-known 
passive  machines.  The  next  step  is  to  utilize  this  capability 
do  develop  resistance  laws  that  closely  match  the  human 
strength  profile.  As  was  shown  here,  it  is  not  possible  to 
match  the  human  strength  curve  using  linear  impedances 
due  to  the  hyperpolic-like  relationship  between  muscle  force 
and  velocity  first  noted  by  Hill  [11].  We  are  currently 
investigating  the  variable  damping  approach  developed  the 
team  at  UC  Berkeley  in  the  1990s  [10]  for  implementation 
on  the  arm  curl  machine,  which  may  require  reconfiguring 
the  controller  in  an  impedance  implementation. 
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Abstract  Transbronchial  biopsy  is  a  common  method  for  tissue 
acquisition  in  diagnosis  of  various  pulmonary  diseases.  However, 
the  procedure  has  technical  limitations  when  performed  on 
peripheral  nodules  due  to  the  small  size  of  peripheral  airways.  To 
enhance  the  accuracy  of  the  procedure,  we  developed  an  electro- 
magnetically  tracked  biopsy  forceps  and  evaluated  the  device  with 
our  image-guided  system.  Our  initial  experimental  results  served 
as  a  proof  of  concept  of  our  system.  Future  work  will  include 
additional  phantom  studies  and  animal  experiments. 

Keywords  Transbronchial  biopsy  •  Electromagnetic  tracking  • 
Virtual  bronchoscopy-  Biopsy  forceps 
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1.  Introduction 

Transbronchial  biopsy  is  a  common  method  used  to  collect  tis¬ 
sue  for  diagnosing  pulmonary  disease  and  for  staging  lung 
cancer.  However,  the  procedure  has  technical  limitations  when 
performed  on  peripheral  nodules  because  bronchoscopes  in 
clinical  use  have  diameters  that  exceed  those  of  the  peripheral 
airways.  Hence,  it  is  difficult  to  reach  a  target  nodule  at  the  lung 
periphery. 

Currently,  pulmonologists  plan  the  transbronchial  biopsy  ap¬ 
proach  by  examining  a  number  of  Computed  Tomography  (CT) 
images  before  the  procedure.  Then,  they  manipulate  a  flexible 
fiberoptic  bronchoscope  into  segmental  and  subsegmental  bronchi 
as  far  as  the  diameter  of  the  bronchoscope  permits.  Finally,  they 
insert  a  biopsy  forceps  through  the  working  channel  of  the 
bronchoscope,  and  perform  the  biopsy  blindly.  Consequently, 
more  than  half  of  the  procedures  fail  to  reach  peripheral  targets 
[1].  When  these  failures  occur,  pulmonologists  must  repeat  the 
procedure  or  follow  up  with  more  invasive  methods,  such  as  CT- 
guided  percutaneous  needle  biopsy  or  surgical  biopsy,  that  have 
increased  complication  rates. 

We  propose  to  overcome  these  limitations  and  enhance  the 
accuracy  of  transbronchial  biopsy  by  developing  a  minimally 
invasive  image-guided  system  that  uses  an  innovative  electro- 
magnetically  tracked  biopsy  forceps.  To  achieve  this  goal,  we 
developed  a  prototype  system  and  electromagnetically  tracked 
forceps  and  we  evaluated  this  system  in  phantom  studies. 

2.  Methods 

2.1  Electromagnetically  tracked  (EM)  biopsy  forceps 
Our  EM  configuration  consists  of  a  field  generator  that  creates 
time  dependent  electromagnetic  fields,  the  five  degree  of  freedom 
sensor  (a  tiny  wire  coil,  0.8  mm  diameter  with  8  mm  length) 
placed  inside  of  the  forceps,  a  control  unit,  and  a  sensor  interface 
unit  that  measures  the  voltage  induced  in  the  coils  by  the  changing 
field.  The  system  can  then  calculate  the  position  and  orientation  of 
the  coil  in  3D  space  relative  to  the  field  generator.  We  used  the 
Aurora  EM  tracking  system  from  Northern  Digital  Inc  (NDI, 
Waterloo,  Ontario,  Canada)  in  this  experiment. 

The  biopsy  forceps  are  from  ConMed  Corporation  (Utica,  NY, 
USA).  The  exterior  metallic  sleeve  of  the  biopsy  forceps  has  been 
replaced  for  a  2  cm  long  region  (Fig.  1)  with  a  hollow  medical 
grade  polyethylene  tube  of  the  same  exterior  diameter  (1.8  mm). 
The  sensor  coil  is  placed  inside  of  this  tube,  close  to  the  jaws,  in  a 
fixed  position  relative  to  the  tube.  The  sensor  axis  is  parallel  with 
the  pulling  wire  of  the  forceps.  The  sensor’s  electrical  wires  go 
through  the  tube  and  forceps  handle,  and  are  connected  to  the 


Fig.  1  Wireframe  model  and  a  photograph  of  the  EM  tracked 
biopsy  forceps 


Aurora  control  unit  through  the  sensor  interface.  The  forceps  can 
be  used  through  a  2  mm  minimum  bronchoscope  working  channel 
like  the  regular  biopsy  forceps. 

2.2  Image-guided  system 

We  constructed  a  prototype  image-guided  system  including  vol¬ 
ume  visualization,  virtual  endoscopy,  electromagnetic  tracking, 
and  registration  [2].  This  system  accomodates  the  forceps  and 
provides  a  user-friendly  interface  to  assist  the  pulmonologist  in 
targeting  these  difficult-to-reach  lesions.  The  system  also  provides 
an  image  overlay  of  the  forceps  within  the  CT  image  volume.  The 
software  provides  the  clinician  with  6  views  (3  multiplanar  views, 
volume  visualization  view,  virtual  bronchoscopy,  and  optical 
bronchoscope  view).  Using  this  image-guided  system,  the  tracked 
forceps  can  be  advanced  to  the  simulated  tumor  site. 

2.3  Experiments 

To  test  the  accuracy  of  the  forceps,  a  phantom  study  was  con¬ 
ducted  in  the  interventional  suite  at  Georgetown  University 
Medical  Center  (Figure  2).  Two  transbronchial  biopsy  simula¬ 
tions  with  a  rubber-made  bronchial  tree  model  were  performed. 
The  workflow  of  the  procedure  was  as  follows: 

1.  Two  target  fiducials  were  placed  on  the  bronchial  tree  of  the 
phantom  as  reference  targets. 

2.  A  CT  scan  was  taken.  A  lung  volume  with  1  mm  axial 
reconstruction  was  acquired. 

3.  The  registration  of  the  CT  space  and  electromagnetic  space 
was  performed  by  first  selecting  four  surface  fiducials  in  the 
CT  images  and  then  touching  each  corresponding  fiducial 
with  the  actively  tracked  forceps. 

4.  We  performed  two  transbronchial  trials  by  advancing  the 
forceps  to  the  target  fiducial  site. 

5.  For  each  biopsy,  a  3D  rotational  angiography  (DynaCT) 
image  was  acquired  to  measure  the  accuracy  of  the  biopsy,  as 
determined  by  the  distance  between  the  biopsy  device  and  the 
target  fiducial. 

3.  Results 

Table  1  shows  the  result  of  our  experiments.  We  performed  two 
DynaCT  scans,  and  the  distances  calculated  from  DynaCT  were 
11.40  mm  (Experiment  1)  and  8.33  mm  (Experiment  2).  Figure  3 
shows  the  visualization  from  the  DynaCT  scan  for  the  second 
experiment.  Respectively,  we  obtained  distances  of  16.78  and 
10.93  mm  from  our  image-guided  software.  The  error  range  was 


Fig.  2  Experiment  Setup.  A  rubber  bronchial  tree  was  put  in  a 
human  body  phantom.  We  used  a  Pentax  bronchoscope  EPM- 
3300  for  the  experiment  and  an  Aurora  EM  tracking  system. 
DynaCT  rotational  angiography  was  used  to  acquire  a  volume  to 
measure  the  distance  from  the  biopsy  forceps  to  the  target  fiducial 
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Fig.  3  Visualization  from  DynaCT  scan  for  the  second  experi¬ 
ment.  The  left  figure  visualizes  the  airway  and  skin  surface,  and 
the  right  figure  visualizes  the  bronchoscope,  biopsy  forceps,  and 
target  fiducial 


Table  1  Summary  of  our  experiments  (units:  mm) 


Distance 

Distance 

Distance 

from  CT  images 

from  DynaCT 

difference 

Experiment  1 

16.78 

11.40 

5.34 

Experiment  2 

10.93 

8.33 

2.60 

2.60-5.34  mm  in  this  experiment.  The  RMS  (root-mean-square) 
registration  errors  were  2.26  and  1.84  mm,  respectively. 

While  these  errors  were  larger  than  we  expected,  the  overall  test 
showed  the  feasibility  of  our  approach.  Possible  reasons  for  the 
error  include  nearby  metal  in  the  interventional  radiology  suite 
and  movement  of  the  airway  model. 


4.  Conclusion 

This  paper  presented  our  initial  experimental  results  of  using  a 
newly  developed  device,  an  electromagnetically  tracked  biopsy 
forceps.  We  customized  an  off-the-shelf  biopsy  forceps  by  placing 
an  EM  sensor  coil  inside  the  biopsy  forceps.  To  test  the  accuracy 
of  the  device,  we  constructed  an  image-guided  system  that  in¬ 
cluded  volume  visualization,  virtual  bronchoscopy  module,  image 
registration,  and  electromagnetic  tracking.  Our  initial  results  show 
that  this  device  may  enhance  the  accuracy  of  transbronchial 
biopsy. 

In  future  work,  we  hope  to  improve  the  accuracy  of  the  tracking 
by  image  registration.  Specifically,  by  using  the  EM  tracking 
system,  the  search  space  of  the  image  registration  algorithm  can 
be  narrowed,  which  may  make  it  possible  to  interactively  perform 
registration  between  the  optical  video  bronchoscpic  images  and 
virtual  bronchoscopic  images.  The  combination  of  EM  tracking 
and  image  registration  algorithm  might  also  compensate  for 
respiratory  motion  during  the  procedure. 
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ABSTRACT 

Vertebroplasty  is  a  minimally  invasive  procedure  in  which  bone  cement  is  pumped  into  a  fractured  vertebral 
body  that  has  been  weakened  by  osteoporosis,  long-term  steroid  use,  or  cancer.  In  this  therapy,  a  trocar  (large 
bore  hollow  needle)  is  inserted  through  the  pedicle  of  the  vertebral  body  which  is  a  narrow  passage  and  requires 
great  skill  on  the  part  of  the  physician  to  avoid  going  outside  of  the  pathway.  In  clinical  practice,  this  procedure 
is  typically  done  using  2D  X-ray  fluoroscopy.  To  investigate  the  feasibility  of  providing  3D  image  guidance,  we 
developed  an  image-guided  system  based  on  electromagnetic  tracking  and  our  open  source  software  platform 
the  Image-Guided  Surgery  Toolkit  (IGSTK).  The  system  includes  path  planning,  interactive  3D  navigation,  and 
dynamic  referencing.  This  paper  will  describe  the  system  and  our  initial  evaluation. 

Keywords:  image  guidance,  vertebroplasty,  electromagnetic  tracking 

1.  INTRODUCTION 

Each  year  osteoporosis  accounts  for  an  estimated  annual  incidence  of  700,000  fractures;  of  these,  approximately 
260,000  are  vertebral  compression  fractures.  Vertebroplasty  is  an  increasingly  popular  medical  procedure  for 
treatment  of  this  condition,  with  14,152  cases  reported  in  the  U.S.  in  2001  and  case  volume  increasing  to 
24,558  in  2003(+73.5%).1  In  conventional  vertebroplasty,  2D  X-ray  projection  images  are  used  to  guide  a  trocar 
consisting  of  a  solid  stylete  and  hollow  cannula  through  the  pedicle  and  into  the  fractured  vertebra.  The  vertebral 
body  is  then  stabilized  by  injecting  bone  sement  through  the  cannula.  This  is  a  difficult  procedure  to  perform, 
since  the  trocar  is  inserted  through  a  relatively  narrow  pedicle  relying  solely  on  2D  X-ray  projection  images.  To 
gain  an  understanding  of  the  underlying  anatomical  structures  and  the  trocar’s  location  physicians  acquire  many 
X-ray  images.  This  exposes  the  patient  to  ionizing  radiation  and  increases  the  physician’s  cumulative  exposure. 
To  overcome  these  drawbacks,  we  propose  to  use  an  image  guidance  system  utilizing  electromagnetic  tracking, 
and  a  pre-operative  cone-beam  CT.  This  system  replaces  the  intermittent  use  of  2D  projection  images  with  a 
dynamic  3D  visualization  of  the  anatomical  structures  and  tracked  instruments. 

We  have  previously  evaluated  the  use  of  image  guidance  for  vertebroplasty.2  In  that  work  we  used  CT  images 
and  electromagnetic  tracking  to  provide  navigation  information.  Registration  of  image  space  to  patient  space 
was  based  on  using  three  tracked  needles  as  registration  fiducials.  The  guidance  system  displayed  the  standard 
axial,  sagittal,  and  coronal  views,  along  with  a  three  dimensional  view.  The  system  was  successfully  used  by  an 
experienced  interventional  radiologist  to  perform  14  needle  insertions  in  seven  vertebrae  of  an  anthropomorphic 
phantom. 

In  this  paper  we  describe  a  new  open  source  version  of  this  software  which  provides  improved  guidance  using 
novel  volume  re-slicing  views  and  a  targeting  view  that  are  tailored  for  needle  insertion  procedures.  We  evaluate 
the  fiducial  localization  accuracy  when  using  needles  as  the  registration  fiducials  and  we  evaluate  the  target 
registration  error  of  our  system  using  an  anthropomorphic  phantom. 
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2.  MATERIALS  AND  METHODS 

We  developed  an  image  guidance  system  for  vertebroplasty  based  on  the  Image-Guided  Surgery  Toolkit  (IGSTK).3 
IGSTK  is  an  open  source  component  based  C++  software  library  which  provides  an  open  framework  to  allow 
researchers  to  develop  customized  image-guided  applications  with  minimal  effort.  We  used  the  following  toolkit 
components:  1)  tracker  interface;  2)  DICOM  image  reader;  3)  paired-point  registration;  and  4)  visualization 
components.  The  library  utilizes  a  state  machine  architecture  for  robustness  and  includes  detailed  error  logging. 
Our  application’s  Graphical  User  Interface  (GUI)  was  developed  using  the  Fast  Light  Toolkit  (FLTK),  which  is 
one  of  the  GUI  toolkits  that  are  compatible  with  IGSTK. 

The  navigation  system  follows  the  common  workflow  in  which  the  physician  imports  a  volumetric  image  data 
set,  plans  the  trocar  insertion  trajectory,  registers  the  image  data  to  the  patient,  and  navigates  to  the  target 
using  tracked  tools  that  are  overlayed  onto  the  image  data. 

All  volumetric  data  was  obtained  in-situ  using  a  cone-beam  CT  system,  the  Axiom  Artis  dFA  (Siemens  AG, 
Medical  Solutions,  Erlangen,  Germany).  Images  are  256x256x221  with  a  0.8mm  isotropic  spacing.  The  system  is 
installed  in  the  interventional  radiology  suite,  and  enables  the  acquisition  of  3D  images  in-situ.  The  data  is  then 
directly  downloaded  to  a  DICOM  server  running  on  the  navigation  system’s  computer.  The  use  of  cone-beam 
CT  does  not  change  any  aspect  of  the  navigation  system,  as  compared  to  the  use  of  standard  CT.  What  it  does 
change  is  the  procedure  workflow,  which  is  greatly  improved,  as  there  is  no  need  to  transport  the  patient  to  the 
CT  suite  and  then  back  to  the  interventional  suite.  In  turn  this  greatly  effects  the  acceptance  of  such  systems 
into  clinical  practice.4 

Tracking  was  performed  with  the  Aurora  system  (Northern  Digital  Inc.,  Waterloo,  Canada).  The  tracked 
tools  included  two  22  gauge  MagTrax  needles  (Traxtal  Inc.,  Toronto,  ON,  Canada).  These  are  five  degree  (5D)  of 
freedom  tools,  with  the  needle  pose  known  up  to  a  rotation  around  the  needle  shaft.  To  perform  the  vertebroplasty 
we  use  a  modified  KyphX  Osteo  Introducer  (Kyphon  Inc.,  Sunnyvale,  CA,  USA),  with  an  embedded  5D  Aurora 
sensor  coil.  Figure  1(a)  shows  these  components. 

All  experiments  were  performed  using  an  anthropomorphic  spine  phantom  comprised  of  a  torso  and  vertebral 
column  (Sawbones  worldwide,  Pacific  Research  Laboratories  Inc.,  Vashon,  Washington,  USA).  The  spine  was 
fixed  in  place  inside  the  torso  using  urethane  foam  as  shown  in  Figure  1(b).  This  was  necessary  as  we  have  empir¬ 
ically  found  that  the  fixation  provided  by  the  phantom’s  structure  is  not  sufficient  for  performing  vertebroplasty. 
Without  this  additional  fixation  the  vertebral  body  exhibited  considerable  motion,  approximately  10mm,  which 
is  beyond  the  motion  observed  in  clinical  interventions. 

We  now  describe  our  novel  user  interface  followed  by  our  needle  based  registration  approach. 

2.1  Graphical  User  Interface 

To  improve  the  quality  of  the  image  guidance  we  have  designed  a  Graphical  User  Interface  (GUI),  that  is  tailored 
for  needle  based  interventions.  Preoperatively,  we  display  the  standard  axial,  sagittal,  and  coronal  views.  The 
physician  manually  identifies  a  target  point  on  the  vertebra  and  an  entry  point  on  the  pedicle.  The  direction 
defined  by  these  two  points  defines  the  desired  path.  Intraoperatively,  once  the  patient  is  registered,  we  provide 
three  re-sliced  views,  axial,  off-axial  and  off-sagittal.  The  off-axial  and  off-sagittal  views  are  re-slice  planes  that 
follow  the  needle  axis.  That  is,  these  are  axial  and  sagittal  views  that  are  tilted  so  that  the  needle  shaft  is  in 
plane.  This  approach  is  based  on  our  observations  of  current  clinical  practice  for  needle  based  biopsies  under 
interventional  CT.  In  these  procedures  the  physician  usually  tries  to  align  the  needle  to  the  fixed  imaging  plane. 
We  provide  similar  views  by  aligning  the  imaging  plane  to  the  needle. 

The  re-slice  plane  is  obtained  as  follows.  Given  the  needle  shaft  direction,  u,  and  needle  tip  location  p,  both 
in  the  image  coordinate  system,  the  off-axial  re-slice  plane  is  defined  as: 

nT(q-p)  =  0 

where  n  =  u  x  x.  Note  that  when  u  x  x  ~  0  we  set  n  =  z.  The  off-sagittal  plane  is  obtained  similarly. 
Figure  2  illustrates  this  concept. 
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(a)  (b) 

Figure  1.  Hardware  components  used  in  the  evaluation  of  the  vertebroplasty  image- guided  navigation  system:  (a)  Elec¬ 
tromagnetic  system  field  generator,  tracked  needles  and  vertebroplasty  trocar,  and  (b)  anthropomorphic  phantom,  torso 
and  vertebral  column. 


Figure  2.  Off  axis  views:  (a)  tilting  the  axial  (xy  plane)  and  (b)  tilting  the  sagittal  (yz  plane)  so  that  the  needle  shaft  is 
in  the  viewing  plane. 


Finally,  we  also  provide  a  targeting  view.  The  target  is  displayed  as  a  green  sphere,  the  needle  tip  as  a  red 
ring,  and  the  needle  hub  as  a  yellow  ring.  To  align  the  needle  to  the  planned  path  the  physician  centers  all  three 
elements.  Once  the  needle  is  aligned  the  physician  advances  the  needle  using  a  depth  gage  which  displays  the 
distance  between  the  needle  tip  and  the  target.  Figure  3  shows  our  GUI. 

2.2  Registration  method 

To  perform  registration  we  use  two  MagTrax  needles.  The  needles  are  inserted  into  the  perispinal  ”  musculature” 
lateral  to  the  spinous  process  of  the  vertebra  of  interest.  This  is  done  percutaneously,  minimizing  trauma  to  the 
patient.  Once  the  needles  are  firmly  lodged  they  form  a  rigid  body  with  the  vertebra.  In  our  system,  this  pair 
of  needles  serves  a  dual  purpose.  They  are  used  as  fiducials  for  registration,  and  once  registration  is  performed 
the  two  5D  needles  serve  as  a  single  6D  dynamic  reference  frame. 

Our  image  to  patient  registration  is  similar  to  the  approach  presented  in.5  It  is  based  on  the  use  of  fiducial 
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Figure  3.  Graphical  user  interface  for  vertebroplasty  guidance.  Axial  view  (top-left),  off-sagittal  view  (top-right),  off-axial 
(bottom-left),  and  targeting  view  (bottom-right). 


location  and  orientation.  This  enables  us  to  use  two  fiducials  instead  of  the  three  required  for  3D/3D  rigid 
registration  when  fiducial  location  is  the  only  input. 

We  next  describe  our  registration  approach  in  detail. 

First,  the  two  needles  are  calibrated.  The  offset  between  the  electromagnetic  coil  embedded  in  the  needle 
shaft  and  the  needle  tip  is  estimated  using  pivot  calibration.6  In  our  case,  the  sensor  coil’s  main  axis  is  aligned 
with  the  needle  shaft,  thus  the  offset  to  the  needle  tip  is  along  a  single  axis. 

Then  the  needles  are  inserted  percutaneously  and  a  cone-beam  CT  data  set  is  acquired.  The  physician 
manually  identifies  the  needle  tip  and  a  point  on  the  needle  shaft  in  the  3D  image.  This  defines  the  shaft 
direction  u  in  the  image  coordinate  system.  We  then  obtain  a  virtual  fiducial  point  on  the  needle  shaft  at  a 
distance  of  20mm  from  the  needle  tip.  Thus,  each  needle  provides  us  with  two  points.  The  choice  of  a  20mm 
offset  from  the  needle  tip  is  based  on  the  physical  characteristics  of  our  needles.  The  needle  shaft  is  only  locally 
rigid  with  respect  to  the  sensor  coil.  As  our  sensors  are  embedded  approximately  10mm  from  the  needle  tip,  we 
define  our  virtual  point  10mm  in  the  opposite  direction  from  the  needle  tip.  This  20mm  segment  around  the 
sensor  coil  does  behave  as  a  rigid  body. 

Finally,  we  position  the  electromagnetic  field  generator  next  to  the  patient,  read  the  sensors’  locations  and 
orientations  and  compute  the  locations  of  the  two  needle  tips  and  two  virtual  points.  Rigid  registration  is  then 
computed  automatically  using  these  four  points  as  input  to  a  quaternion  based  analytic  solution.7  Figure  4 
illustrates  the  registration  point  identification  in  image  space  and  in  physical  space. 

3.  EXPERIMENTAL  RESULTS 

Given  that  our  navigation  system  uses  an  electromagnetic  tracking  system  we  first  need  to  assess  its  accuracy  in 
the  specific  interventional  environment.  We  have  previously  found  that  accuracy  is  highly  dependent  upon  the 
interventional  environment.8  In  that  study  we  also  evaluated  the  Aurora  system  in  the  interventional  radiology 
environment.  We  concluded  that  the  system’s  accuracy  is  sufficient  for  our  procedure,  with  a  median  positional 
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Figure  4.  In  image  space,  left  coordinate  system,  the  user  manually  identifies  needle  tip  and  needle  shaft  (x  marks)  and 
the  registration  points  (circles)  are  computed.  In  physical  space,  right  coordinate  system,  the  location  of  the  sensor  coil 
is  obtained  and  the  registration  points  (circles)  are  computed  using  the  offset  1  obtained  via  pivot  calibration. 


data  set  1 

data  set  2 

tip 

{p'x  i  Oy  i  & z) 

shaft 

(&X1  Oyi  ®z) 

tip 

(Te5  OyiOz) 

shaft 

{p'x  i  ®y  5  & z) 

needle  1 
needle  2 

(0.29,  0.75,  0.24) 
(0.3,  1.2,  0.61) 

(0.5,  0.82,  0.39) 
(0-87,  1.3,  0.74) 

(0.39,  1,  0.39) 
(0.43,  0.83,  0.4) 

(0.48  1.1  0.97) 
(0.54,  0.87,  0.55) 

Table  1.  Standard  deviations  for  fiducial  localization,  as  identified  by  three  operators.  The  fiducials  are  the  needle  tip, 
and  a  virtual  fiducial  located  20mm  away  from  the  tip  on  the  needle  shaft.  All  measurements  are  in  millimeters. 


error  of  0.53mm  and  standard  deviation  of  0.43mm.  This  accuracy  was  obtained  when  the  C-arm  based  cone- 
beam  CT  system  was  away  from  its  imaging  position,  which  is  the  setup  in  which  our  navigation  system  is 
used. 

An  important  aspect  of  any  image  guidance  system  that  uses  fiducials  is  the  fiducial  localization.  In  our 
application,  we  are  using  tracked  fiducials,  which  enable  automatic  fiducial  localization  in  physical  space.  In 
image  space  we  manually  identify  the  fiducials  in  the  image.  In  our  case  the  needle  tip  is  well  defined,  but  the 
point  on  the  needle  shaft  is  a  virtual  fiducial  and  is  expected  to  have  a  higher  localization  variability.  It  should 
be  noted  that  the  localization  of  the  virtual  fiducial  is  dependent  on  the  localization  of  the  needle  shaft  and 
needle  tip.  That  is,  if  the  needle  shaft’s  true  location  is  identified  the  virtual  fiducial  localization  will  have  the 
same  variability  as  the  needle  tip  localization. 

To  assess  the  variability  of  fiducial  localization  in  the  images  we  performed  the  following  experiment.  The 
MagTrax  needles  were  inserted  into  the  phantom  next  to  the  vertebra  of  interest  and  a  cone-beam  CT  data  set 
was  acquired.  We  acquired  two  such  data  sets.  We  then  analyzed  the  possible  localization  variability  by  having 
three  engineers  localize  the  fiducials  (four  needle  points)  ten  times  in  each  of  the  images. 

While  the  needle  tip  is  well  defined,  the  virtual  fiducial  on  the  needle  shaft  is  indirectly  identified  by  the 
operator  by  marking  a  point  on  the  needle  shaft.  We  thus  expected  that  the  variability  in  the  virtual  fiducial 
localization  would  be  greater  than  needle  tip  localization.  When  analyzing  the  data  we  observed  that  the 
variability  in  virtual  fiducial  localization  is  comparable  to  needle  tip  localization,  albeit  with  a  slightly  greater 
variability.  Table  1  summarizes  this  experiment. 

The  complete  system  was  then  evaluated  using  the  anthropomorphic  phantom  described  above.  To  provide 
well  defined  targets  we  embedded  2 mm  steel  ball  bearings  into  the  vertebral  column  in  clinically  viable  locations 
that  correspond  to  valid  trocar  positions.  The  system  was  then  used  to  perform  navigated  vertebroplasty. 
Figure  5(a)  shows  our  experimental  setup. 

To  evaluate  the  accuracy  of  our  system  and  the  possible  procedure  variability  we  had  a  second  year  medical 
student,  a  fellow,  and  an  attending  physician  each  perform  five  navigated  interventions.  An  intervention  was 
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(a)  (b) 

Figure  5.  In-vitro  experimental  setup  in  the  interventional  radiology  suite,  and  a  lateral  projection  image  acquired  after 
successful  navigation  which  shows  the  trocar  inside  the  pedicle  with  its  tip  near  the  target  ball  bearing. 


Experiment 

student 

fellow 

attending 

1 

2 

3 

4 

5 

mean(std) 

1.3 

1.44 

2.28 

2.36 

2.62 

2.0(0.59) 

0.94 

0.96 

1.2 

1.53 

1.75 

1.28(0.36) 

0.6 

1.47 

1.6 

1.63 

1.66 

1.39  (0.45) 

Table  2.  Accuracy  results  for  in-vitro  navigated  vertebroplasty  procedure.  Distances  were  measured  between  trocar  tip 
and  embedded  ball  bearing  using  cone-beam  CT  images.  All  measurements  are  in  millimeters. 


judged  as  a  success  if  the  trocar  was  completely  inside  the  pedicle,  otherwise  it  was  considered  a  failure.  Fig¬ 
ure  5(b)  shows  a  lateral  image  used  for  qualitative  evaluation.  We  then  quantitatively  evaluated  the  accuracy  of 
a  successful  intervention  as  the  distance  between  the  trocar  tip  and  the  target  ball  bearing  center.  This  distance 
was  measured  using  a  confirmation  cone-beam  CT  scan  after  each  trocar  insertion. 

All  procedures  in  this  experiment  were  performed  successfully  by  all  participants.  That  is,  the  trocar  did  not 
breach  the  pedicle.  Not  surprisingly,  the  difference  between  the  medical  student  and  practicing  physicians  was 
reflected  by  the  accuracy  results.  The  fellow  and  attending  exhibit  similar  accuracy  with  the  medical  student 
being  less  accurate.  Table  2  summarizes  this  experiment. 

4.  DISCUSSION  AND  CONCLUSIONS 

We  have  described  an  image- guided  navigation  system  based  on  come-beam  CT  and  electromagnetic  tracking. 
This  system  provides  a  fully  integrated  in-situ  navigation  solution  for  vertebroplasty,  with  3D  image  acquisition 
seamlessly  integrated  with  the  navigation  system.  The  software  is  based  on  the  open  source  Image-Guided 
Surgery  Toolkit  (IGSTK).9 

We  evaluated  the  variability  of  fiducial  localization  in  the  3D  images.  This  was  primarily  motivated  by  the 
fact  that  our  fiducials  are  needles  and  that  we  use  a  virtual  point  on  the  needle  shaft  that  is  indirectly  identified  by 
the  user.  From  analyzing  the  variability  of  three  engineers  performing  this  task  we  conclude  that  the  localization 
variability  is  on  the  order  of  1  mm  which  is  on  the  same  scale  as  our  voxel  sizes  (0.8mm3). 

We  then  evaluated  the  accuracy  of  our  system  using  an  anthropomorphic  phantom.  Three  medical  profession¬ 
als  with  different  skill  levels  (student,  fellow,  attending),  performed  the  procedure  five  times.  All  procedures  were 
deemed  successful  with  the  trocar  inserted  through  the  pedicle  without  breaching  it.  The  variability  due  to  skill 
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level  was  reflected  by  the  distance  between  the  trocar  tip  and  planned  target  point,  with  the  more  experienced 
operators  being  more  accurate. 

To  improve  these  results,  a  semi-automatic  method  for  needle  localization  is  needed,  removing  the  dependence 
of  fiducial  localization  on  the  human  operator.  The  next  level  of  system  validation  should  then  be  done  on 
cadavers. 
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ABSTRACT 

The  goal  of  this  project  is  to  develop  a  robotic  system  to  assist  the  physician  in  minimally  invasive  ultrasound 
interventions.  In  current  practice,  the  physician  must  manually  hold  the  ultrasound  probe  in  one  hand  and  manipulate  the 
needle  with  the  other  hand,  which  can  be  challenging,  particularly  when  trying  to  target  small  lesions.  To  assist  the 
physician,  the  robot  should  not  only  be  capable  of  providing  the  spatial  movement  needed,  but  also  be  able  to  control  the 
contact  force  between  the  ultrasound  probe  and  patient.  To  meet  these  requirements,  we  are  developing  a  prototype 
system  based  on  a  six  degree  of  freedom  parallel  robot.  The  system  will  provide  high  bandwidth,  precision  motion,  and 
force  control.  In  this  paper  we  report  on  our  progress  to  date,  including  the  development  of  a  PC-based  control  system 
and  the  results  of  our  initial  experiments. 

Keywords:  abdominal  procedures,  ultrasound  guidance,  medical  robot,  force  control 


1.  INTRODUCTION 

Ultrasound  is  routinely  used  for  minimally  invasive  interventions  such  as  biopsy  to  obtain  a  tissue  sample  for 
pathological  analysis  [1].  While  typically  these  procedures  can  be  completed  satisfactorily,  there  are  times  when  it  can  be 
difficult  to  precisely  place  the  needle  due  to  respiration.  Finding  a  direct  route  to  the  target  site  while  avoiding  ribs  and 
other  obstacles  can  also  be  problematic.  Small  targets  in  particular  can  be  difficult  to  hit.  Finally,  a  successful  biopsy  also 
depends  greatly  on  the  hand-eye  coordination  of  the  physician,  since  they  must  hold  the  ultrasound  probe  in  one  hand 
and  manipulate  the  needle  with  the  other  hand.  The  probe  must  be  held  still  both  while  injecting  local  anesthesia  and  for 
the  subsequent  procedure  needle  which  must  closely  follow  the  local  anesthesia  tract.  Many  other  ultrasound  guided 
procedures  require  two  hands  for  manipulation  of  needles  and  other  instruments.  These  procedures  include  placement  of 
drainage  catheters,  fiducial  markers  for  radiotherapy,  and  injection  of  agents  such  as  thrombin  or  ethanol.  Further,  if  a 
biopsy  sample  or  aspirate  requires  handling  such  as  making  a  cytologic  slide  it  must  be  passed  off  to  another  individual 
or  the  ultrasound  probe  released  requiring  a  second  localization  for  additional  samples.  Often  an  assistant  is  needed  to 
accomplish  these  procedures. 

For  these  reasons,  we  are  developing  a  robotic  system  to  hold  the  ultrasound  probe  with  the  proper  orientation  and  force 
against  the  abdomen.  This  system  is  intended  to  assist  the  physician  with  accurate  placement  of  the  needle  in  abdominal 
ultrasound-guided  interventions.  The  system  is  based  on  an  existing  six  degree  of  freedom  (DOF)  parallel  robot  called 
SMARTee  [2].  This  robot  includes  a  force/torque  sensor  and  can  be  used  in  both  position  control  and  force  control 
applications. 

Other  researchers  have  investigated  robotics  for  ultrasound  interventions.  Salcudean  and  colleagues  at  the  University  of 
British  Columbia  in  Canada  developed  a  robot-assisted  system  for  ultrasound  diagnosis  [3].  The  system  consisted  of  a 
master  hand  controller,  a  slave  manipulator  that  held  the  probe,  and  a  computer  control  system.  The  system  could  be 
used  both  locally  and  in  a  teleoperated  mode. 
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Hong  and  colleagues  at  the  University  of  Tokyo  developed  an  ultrasound  based  needle  insertion  robot  for  percutaneous 
cholecystostomy  (drainage  of  the  gall  bladder).  The  robot  included  a  5 -DOF  passive  arm  to  position  the  needle  at  the 
skin  entry  point  and  a  2-DOF  needle-drive  mechanism  for  automatic  needle  insertion.  A  spring  mechanism  was  used  to 
maintain  the  proper  pressure  on  the  skin  to  ensure  good  ultrasound  images  could  be  obtained. 

A  master-slave  system  for  ultrasound  teleoperation  has  also  been  constructed  by  Troccaz  and  colleagues  in  Grenoble, 
France  [4].  The  slave  robot  is  a  cable-driven  non-rigid  structure  which  consists  of  a  parallel  positioning  mechanism  and  a 
serial  orientation  mechanism.  The  master  robot  is  a  PHANToM  device  (Sensable  Technologies  Inc.)  which  provides 
force  feedback. 

In  addition  to  the  three  groups  mentioned  here,  other  researchers  have  proposed  similar  devices.  However,  to  the  best  of 
the  authors’  knowledge,  none  of  these  devices  have  been  commercialized.  Therefore,  we  believe  there  is  still  a  need  for 
additional  research  in  this  area.  In  this  paper,  we  present  our  work  in  developing  a  small  parallel  manipulator  with  force 
control  for  robotically  assisted  ultrasound  procedures. 


2.  METHODS 


2.1  Requirements 

The  hardware  requirements  for  the  robot  were  generated  through  a  literature  review  for  parameters  that  determine 
ultrasound  image  quality.  Salcudean  and  colleagues  measured  the  ultrasound  probe  movements  and  forces  during  carotid 
artery  examinations  [5].  The  x,  y,  and  z  translational  motions  were  estimated  at  130  mm,  150  mm,  and  100  mm 
respectively.  Orientations  about  the  x,  y,  and  z  axes  were  estimated  as  35  to  -50  degrees,  ±75  degrees,  and  ±45  degrees. 
They  also  measured  average  velocities  to  be  small,  on  the  order  of  5  mm/sec  and  3°/sec.  The  x,  y,  and  z  probe  forces 
were  also  measured  and  estimated  as  3.8N,  4.2N,  and  6.4N  as  sufficient  for  high  ultrasound  image  quality. 

The  literature  review  provided  a  clear  indication  of  basic  hardware  requirements.  In  addition  to  this,  respiratory  motion 
requires  a  force  control  ability  so  that  the  contact  force  between  the  probe  and  patient  skin  can  be  maintained.  A 
successful  implementation  of  the  proposed  system  must  therefore  ideally  have  a  bandwidth  greater  than  100Hz  for  force 
control.  Since  the  workspace  requirements  were  sufficiently  small,  the  choice  between  a  parallel  or  serial  mechanism 
robot  tended  towards  a  system  that  could  potentially  be  more  accurate.  These  derived  requirements  were  found  to  lie 
within  the  capabilities  of  an  existing  parallel  mechanism  robotic  manipulator,  called  the  SMARTee  system.  The 
SMARTee  system  was  the  result  of  research  into  a  novel  parallel  mechanism  manipulator  with  all  revolute  joints.  The 
specifications  of  this  system  were  found  to  satisfy  those  deduced  from  the  literature  review.  To  reduce  development  time 
and  to  keep  prototyping  cost  to  a  minimum,  it  was  decided  to  retrofit  this  robot  to  manipulate  the  ultrasound  probe.  The 
requirements  deduced  from  the  literature  review  and  corresponding  system  specifications  for  the  SMARTee  system  are 
summarized  in  Table  1. 

Table  1:  Summary  of  requirements  for  ultrasound  diagnosis  and  comparable  specifications  of  the  SMARTee  system. 


Specification 

Ultrasound  diagnosis 

SMARTee  system 

Workspace 

lOOxlOOxlOOmm3 

90x90xl00mm3 

Roll  angle  range 

(-30,30)  degrees 

(-30,30)  degrees 

Yaw  angle  range 

(-30,30)  degrees 

(-30,30)  degrees 

Pitch  angle  range 

(-30,30)  degrees 

(-90,90)  degrees 

Force  control  bandwidth 

>  100Hz 

1000Hz 

2.2  System  architecture 
Mechanical  system  setup 

The  SMARTee  parallel  mechanism  system  consists  of  a  base  plate  and  top  plate  connected  together  by  three  legs.  Each 
leg  consists  of  three  joints  and  provides  six  degrees  of  freedom,  a  simplified  diagram  of  operation  is  provided  in  Figure 
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la.  Each  leg  is  actuated  by  two  brushed  motors  (Model  No.:  D1H18-18-001Z,  BEI  Motion  Systems  Co.,  Goleta,  CA, 
USA)  and  Hall  Effect  sensors  are  used  as  safety  limit  switches  to  sense  the  travel  on  each  leg.  A  full  description  of  the 
kinematic  model  is  provided  in  [2]. 

Forces  applied  at  the  top  plate  are  sensed  by  a  6-DOF  force/torque  sensor  from  ATI  Industrial  Automation  Inc.  (ATI-IA, 
Apex,  NC,  USA).  The  sensor  offers  high  signal-to-noise  ratio  and  provides  a  force  sense  limit  of  301bf  and  lOOlbf  along 
the  X,  Y  and  Z  axes  respectively.  Torque  sense  limit  is  lOOlb-in  along  all  axes.  A  portable  ultrasound  device,  Terason 
2000  model  (Teratech,  Burlington,  MA,  USA)  used  in  conjunction  with  the  4C2  model  transducer  (curved  linear,  128 
element  probe)  is  used  for  image  acquisition.  The  Terason  was  chosen  for  its  lightweight,  portable  nature,  and  ease  of 
use.  The  ultrasound  probe  is  rigidly  mounted  to  the  ATI  sensor  using  a  straight  tube  linkage,  so  that  the  forces  applied  to 
the  probe  head  can  be  directly  measured.  Figure  lb  shows  the  retrofitted  SMARTee  system  with  the  ultrasound  probe 
holder  and  sensors  in  place. 


Figure  lb:  SMARTee  robot  retrofitted  to  hold  the  ultrasound 

probe 


Control  system  setup 

The  original  SMARTee  configuration  relied  on  a  VME  architecture  running  the  VX Works  operating  system  (Wind  River 
Systems,  Alameda,  CA,  USA)  utilizing  a  GreenSpring  motion  control  board  and  servo  amplifiers  from  Advanced 
Motion  Control.  The  control  system  was  updated  to  a  PC-based  controller  system  to  bring  the  system  up  to  date  and 
make  the  system  easier  to  program  and  use. 

The  PC-based  system  consists  of  a  Galil  6-axis  Accelera  series  motion  controller  card  (Galil  Motion  Control,  Rocklin, 
CA,  USA)  that  communicates  to  the  host  PC  over  the  PCI  bus.  The  control  system  architecture  is  shown  in  Figure  2.  The 
left  side  depicts  the  control  PC  which  communicates  with  the  ATI  force  sensor  control  box  through  serial  (RS-232) 
connection.  The  encoders  and  Hall  Effect  sensors  within  the  SMARTee  robot  communicate  with  the  Galil  motion  control 
card  to  determine  motor  position  and  limit  range  respectively.  The  AMC  servo  amplifiers  from  the  old  system 
architecture  were  reused  to  power  the  motors,  but  with  the  Galil  motion  card  communicating  with  the  servo  amplifiers 
for  kinematic  calculations  and  trajectory  planning. 
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Figure  2:  PC  based  robot  control  system 


System  software  architecture 

The  long-term  software  implementation  for  the  SMARTee  system  was  intended  to  be  built  using  open  source  resources 
and  run  on  a  real-time  Linux  operating  system.  As  this  is  the  first  system  iteration  for  ultrasound  interventions  and  the 
purpose  was  largely  to  assess  the  feasibility  of  this  approach  using  the  SMARTee  hardware,  it  was  decided  to  build  the 
first  software  release  of  the  system  on  Microsoft  Windows  to  save  development  time.  This  was  done  as  the  authors  were 
more  familiar  with  the  Windows  platform,  although  a  real-time  platform  would  be  more  suitable  for  clinical  applications. 

2.3  Robot  kinematics 

The  notation  used  to  describe  the  SMARTee  kinematics  is  shown  in  Figure  3  and  was  described  in  [2].  The  fixed  global 
reference  frame  is  called  the  base  frame  (Xb,  Yb,  Zb)  and  is  located  at  the  bottom  center  of  the  base  plate.  The  z-axis  is 
perpendicular  to  the  base  plate,  the  x-axis  is  in  the  plane  of  the  bottom  of  the  base  plate  and  is  directed  toward  the 
universal  joint  on  the  first  leg,  and  the  y-axis  is  given  by  the  right  hand  rule.  The  base  frame  is  offset  a  distance  Zbase  offset 
from  the  plane  containing  the  centers  of  the  three  universal  joints.  Another  reference  frame,  called  the  top  frame  (Xt,  Yt, 
Zt)  is  located  at  the  top  center  of  the  top  plate.  The  z-axis  is  perpendicular  to  the  top  plate,  the  x-axis  is  in  the  plane  of  the 
top  of  the  top  plate  and  is  directed  toward  the  spherical  joint  on  the  first  leg,  and  the  y-axis  is  given  by  the  right  hand  rule. 
The  top  frame  is  offset  a  distance  Ztop  0ffset  from  the  plane  containing  the  centers  of  the  three  spherical  joints. 
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Figure  3:  Kinematic  model  of  SMARTee  robot  [2] 


The  inverse  kinematics  for  parallel  manipulators  can  typically  be  described  in  closed  form  and  the  forward  kinematics 
are  typically  given  numerically  (the  opposite  is  true  for  serial  manipulators  where  the  forward  kinematics  are 
straightforward  and  the  inverse  kinematics  are  more  difficult).  The  inverse  kinematics  problem  for  SMARTee  can  be 
stated  as:  Given  the  position  and  orientation  of  the  top  plate,  compute  the  joint  angles  on  each  of  the  legs.  A  closed  form 
solution  is  available  which  yields  the  first  two  angles  on  the  driven  joints  of  each  leg.  The  forward  kinematics  problem 
can  be  stated  as:  Given  the  two  driven  joint  angles  on  each  leg,  compute  the  position  and  orientation  of  the  top  plate.  As 
noted  above,  there  usually  is  not  a  closed  form  solution  for  the  forward  kinematics  of  a  parallel  manipulator.  A  numerical 
solution  was  developed  which  involves  solving  a  set  of  three  nonlinear  equations. 


2.4  Impedance  control  algorithm 

The  impedance  controller  was  first  introduced  by  Hogan  et  al  [6]  and  has  since  been  used  commonly  in  robot  control 
including  the  admittance  configuration  used  here.  Since  an  impedance  control  algorithm  works  best  when  working  with 
compliant  surfaces,  we  chose  to  use  this  method  to  control  the  SMARTee  end  effector.  In  our  application,  encoders 
mounted  on  the  motor  shafts  read  the  angles  of  the  motors  and  are  then  converted  to  joint  angles.  The  angles  are  then 
sent  to  a  forward  kinematics  module  inside  the  controller  to  determine  the  position  of  the  tool  tip.  Meanwhile,  the  force 
sensor  mounted  on  the  end-effector  will  measure  the  contact  force  between  the  robot  and  patient.  The  desired  move  is 
computed  by  the  impedance  controller  as  a  function  of  the  contact  force  readings  and  current  robot  position.  The 
flowchart  shown  in  Figure  4  shows  the  data  flow  for  the  impedance  controller.  The  impedance  controller  coefficients 
can  been  identified  and  defined  in  advance  to  implement  a  desired  performance. 


Figure  4:  Impedance  control  data  flow 
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3.  RESULTS 


Experimental  verification  of  the  functionality  of  the  modified  SMARTee  system  was  conducted  through  the  course  of 
three  tests.  The  first  test  was  performed  on  an  ultrasound  phantom  developed  at  our  lab  [7].  The  phantom  was  placed 
directly  in  front  of  the  robotic  system  and  the  robot  commanded  to  apply  a  steady  pressure  of  0.61b.  The  experimental 
setup  is  shown  in  Figure  5a  and  an  image  acquired  with  the  Terason  probe  with  the  internal  structures  within  the 
phantom  clearly  visible  is  shown  in  Figure  5b.  Figure  6  provides  a  plot  of  the  contact  force,  as  measured  by  the  ATI 
sensor,  as  the  robot  automatically  adjusts  to  apply  a  constant  force  against  the  phantom.  The  impedance  controller  is  able 
to  stabilize  the  contact  force  in  just  less  than  ten  seconds.  The  successful  results  from  this  test  provided  early  validation 
that  the  retrofitted  SMARTee  robot,  in  conjunction  with  the  impedance  control  algorithm,  is  viable  for  ultrasound 
imaging. 


Figure  5a:  Experiment  setup  Figure  5b:  Image  of  phantom  from  ultrasound  probe 


Force  vs  Time  plot  of  contact  force  at  Ultrasound  probe  head 


Time  (s) 

Figure  6:  Applied  contact  force  as  the  robot  moves 

The  second  test  was  conducted  to  test  the  reliability  of  the  system  under  the  influence  of  a  varying  contact  force  due  to 
respiratory  motion.  An  anthropomorphic  phantom  was  connected  to  a  variable  respirator  [8]  as  shown  in  Figure  7. The 
phantom  was  developed  by  the  URobotics  Faboratory  at  Johns  Hopkins  Medical  Institutions.  The  robotic  end-effector 
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with  the  ultrasound  probe  was  positioned  to  apply  a  fixed  pressure  on  the  phantom.  The  phantom  was  then  made  to 
respirate  at  a  rate  of  20  breaths/min  and  the  robot  was  able  to  successfully  maintain  the  contact  force. 

The  reading  from  the  force  sensor  once  the  robot  had  stabilized  is  shown  in  Figure  8a.  There  is  a  variability  of  about 
0.21b  in  the  contact  force  applied  by  the  robot  during  respiration,  which  was  deemed  acceptable  for  this  application.  The 
respiratory  rate  was  extrapolated  from  the  force  data  by  examining  the  power  spectral  density  plot  as  shown  in  Figure  8b. 
The  largest  frequency  peak  exists  at  0.24Hz,  thereby  implying  a  respiratory  rate  of  14  breaths/min,  which  was  lower  than 
expected.  This  discrepancy  will  be  investigated  further  in  future  work. 


Respiration 
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Robot  \ 
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Figure  7:  Experimental  setup  with  the  anthropomorphic  phantom,  respiratory  pump,  and  robot 


Contact  Force  Plot  Under  Impedance  Control 

0.8^ 


0  . 

0  5  10  15  20 

Time  (s) 

Figure  8a:  Comparison  of  contact  force  with  and  without 
impedance  control 


Figure  8b:  Power  spectral  plot  to  find  dominant 
frequency  in  force  data 
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4.  CONCLUSIONS 


Robotically  assisted  ultrasound  has  the  potential  to  improve  the  physician’s  ability  to  biopsy  difficult  targets  in  the 
abdomen.  The  force-controlled  parallel  mechanism  system  used  here  has  shown  to  be  a  good  architecture  for  this 
application.  This  iteration  used  the  existing  ATI  sensor  and  controller  to  relay  force  and  torque  information  from  the 
sensor  to  the  host  PC  via  serial  communication.  Future  iterations  could  include  a  Measurement  Computing  DAQ  card 
used  in  conjunction  with  the  Comdi  open-source  toolkit  for  digital  I/O  which  would  considerably  speed  up 
communications  with  the  sensor. 
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This  paper  presents  an  overview  of  the  image-guided 
surgery  toolkit  (IGSTK).  IGSTK  is  an  open  source  C++ 
software  library  that  provides  the  basic  components 
needed  to  develop  image-guided  surgery  applications.  It 
is  intended  for  fast  prototyping  and  development  of  image- 
guided  surgery  applications.  The  toolkit  was  developed 
through  a  collaboration  between  academic  and  industry 
partners.  Because  IGSTK  was  designed  for  safety-critical 
applications,  the  development  team  has  adopted  light¬ 
weight  software  processes  that  emphasizes  safety  and 
robustness  while,  at  the  same  time,  supporting  geograph¬ 
ically  separated  developers.  A  software  process  that  is 
philosophically  similar  to  agile  software  methods  was 
adopted  emphasizing  iterative,  incremental,  and  test- 
driven  development  principles.  The  guiding  principle  in 
the  architecture  design  of  IGSTK  is  patient  safety.  The 
IGSTK  team  implemented  a  component-based  architecture 
and  used  state  machine  software  design  methodologies  to 
improve  the  reliability  and  safety  of  the  components.  Every 
IGSTK  component  has  a  well-defined  set  of  features  that 
are  governed  by  state  machines.  The  state  machine 
ensures  that  the  component  is  always  in  a  valid  state  and 
that  all  state  transitions  are  valid  and  meaningful.  Realizing 
that  the  continued  success  and  viability  of  an  open  source 
toolkit  depends  on  a  strong  user  community,  the  IGSTK 
team  is  following  several  key  strategies  to  build  an  active 
user  community.  These  include  maintaining  a  users  and 
developers'  mailing  list,  providing  documentation  (applica¬ 
tion  programming  interface  reference  document  and 
book),  presenting  demonstration  applications,  and  deliver¬ 
ing  tutorial  sessions  at  relevant  scientific  conferences. 

KEY  WORDS:  Image-guided  surgery,  open  source, 
visualization,  registration,  tracking  and  agile 
software  development 

INTRODUCTION 

Minimally  invasive  procedures  are  becoming 
increasingly  popular  in  today’s  healthcare 
system.  Patients  prefer  these  procedures  to  open 


surgeries  because  they  cause  less  trauma  to  the 
body  and  result  in  faster  recovery  times.  Interven¬ 
tional  radiologists  and  surgeons  are  also  becoming 
more  experienced  and  comfortable  performing 
these  procedures.  Interventional  radiologists  use 
instruments  such  as  needles  and  catheters  to 
perform  diagnostic  and  therapeutic  procedures 
guided  by  images.  Examples  of  diagnostic  proce¬ 
dures  include  biopsy  of  suspicious  lesions  and 
injection  of  contrast  agents  for  computed  tomog¬ 
raphy  (CT)  angiography.  Therapeutic  procedures 
include  using  stents  to  open  clogged  arteries  and 
radiofrequency  ablation  techniques  to  destroy 
tumor  tissues. 

In  image-guided  surgery  procedures,  the  surgi¬ 
cal  instruments  are  placed  through  incisions 
guided  by  preoperative  images.  To  track  the 
position  of  the  surgical  instruments,  a  tracking 
system  is  used.  The  tracking  system  shows  the 
position  and  orientation  of  the  surgical  instrument 
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in  the  context  of  preoperative  images.  The  preop¬ 
erative  image  is  aligned  with  the  patient  coordinate 
system  using  a  technique  called  registration.  The 
tracking  system  is  used  to  track  the  position  of  the 
surgical  instrument.  A  typical  image-guided  sys¬ 
tem  is  shown  in  Figure  1. 

The  use  of  open  source  software  for  medical 
procedures  like  image-guided  surgeries  is  gaining 
more  acceptances  in  recent  years.  With  US 
government  agencies  such  as  the  National  Insti¬ 
tutes  of  Health  (NIH)  and  the  National  Science 
Foundation  encouraging  open  source  software 
development,  more  research  groups  are  developing 
and  releasing  software  as  open  source.  The 
Visualization  Toolkit  (VTK)  and  the  Insight 
Toolkit  (ITK)  are  prime  examples  of  the  success 
of  open  source  software  projects.  VTK1,  originally 
developed  at  GE  Global  Research  and  now 
supported  by  Kitware,  provides  a  wide  range  of 
algorithms  in  computer  graphics,  image  process¬ 
ing,  and  visualization.  The  toolkit  has  a  wide  user 
base  with  thousands  of  users  worldwide.  VTK 
provides  advanced  multidimensional  visualization 
algorithms  and  modeling  techniques  such  as 
implicit  modeling,  polygon  reduction,  mesh 
smoothing,  cutting,  contouring,  Delaunay  triangu¬ 
lation,  and  parallel  computing. 

ITK2,  developed  under  the  support  of  the  US 
National  Library  of  Medicine  (NLM)  at  the  NIH, 
provides  advanced  registration  and  segmentation 
algorithms.  The  toolkit  contains  state-of-the-art 
algorithms  and  continues  to  be  updated  and 
maintained  with  supports  from  NLM.  Similar  to 
VTK,  the  software  is  implemented  in  the  C++ 


programming  language  and  provides  wrappers  for 
Tel,  Python,  and  Java  interpreters. 

The  composition  of  the  ITK  development  team 
demonstrates  the  strength  of  a  collaborative  effort 
that  is  greatly  emphasized  in  an  open  source 
project.  The  initial  development  team  consisted  of 
three  commercial  partners  (GE  Corporate  R&D, 
Kitware,  and  MathSoft)  and  three  academic  part¬ 
ners  (University  of  Tennessee,  University  of  North 
Carolina,  and  University  of  Pennsylvania).  Other 
developers  have  since  joined  the  effort,  and  to 
date,  over  50  developers  have  directly  contributed 
code  to  the  toolkit. 

Several  benefits  have  contributed  to  the  in¬ 
creased  popularity  and  acceptance  of  open  source 
software.  Some  of  the  main  benefits  are  listed 
below. 

1)  Open  source  software  encourages  collaboration 
between  academic,  commercial,  and  govern¬ 
ment  institutions  by  providing  a  common  soft¬ 
ware  base  and  creating  a  sense  of  community. 

2)  Open  source  software  saves  resources  by 
avoiding  “reinventing  the  wheel.”  Oftentimes, 
a  researcher  wastes  valuable  time  and  resources 
implementing  basic  infrastructure.  Using  open 
source  implementations  of  basic  functionality, 
researchers  are  able  to  focus  on  new  research 
efforts. 

3)  Open  source  software  provides  a  valuable 
resource  for  educational  purposes.  The  best 
way  to  learn  software  development  is  by 
studying  how  other  developers  have  done  it 
and  by  attempting  to  improve  it. 


Fig  1 .  Image-guided  system  for  brain  surgery  showing  the  optical  tracking  system  at  the  top  right  and  the  display  overlay  on  the  far 
left.  The  patient  is  under  the  blue  cover  in  the  middle.  (Photograph  courtesy  of  Richard  Bucholz,  MD,  St.  Louis  University). 
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4)  Open  source  software  provides  implementation 
of  reference  or  benchmark  algorithms  for 
validation  and  verification  purposes. 

5)  Open  source  software,  when  distributed  under 
the  appropriate  license,  permits  users  to  try  out 
the  technology  without  restrictions,  and  this 
ultimately  encourages  rapid  dissemination  of 
the  technology  and  the  growth  of  a  user 
community. 

With  the  above  benefits,  open  source  toolkits 
are  making  the  transition  into  mission-critical 
applications.  Forrester’s  study3  has  shown  that 
companies  are  rapidly  expanding  their  use  of  open 
source  software  from  simple  applications  (such  as 
email)  to  mission  critical  applications  (such  as 
customer  contact  applications).  In  the  medical 
field,  software  applications  developed  using  open 
source  toolkits  have  been  submitted  to  the  Food 
and  Drug  Administration  (FDA)  for  approval  in 
clinical  studies.  Although  the  FDA  does  not  have 
any  specific  policy  on  open  source  software,  the 
FDA  requires  that  software  included  in  medical 
devices  should  be  developed  following  a  process 
for  which  a  quality  control  system  is  in  place.  As 
one  example,  the  FDA  recently  approved  a  single 
center  clinical  trial  for  an  electromagnetically 
tracked  lung  biopsy  application  developed  using 
image-guided  surgery  toolkit  (IGSTK)  that  will 
begin  shortly  at  Georgetown  University  Medical 
Center. 

IGSTK,  the  focus  of  this  paper,  is  an  open 
source  software  project  developed  with  support 
from  the  National  Institute  of  Biomedical  Imaging 
and  Bioengineering  (NIBIB)  at  NIH.  It  is  a  cross¬ 
platform  C++  library  that  provides  the  basic 
components  necessary  to  develop  an  image-guided 
system.  It  is  intended  for  fast  prototyping  and 
development  of  robust  image-guided  applications. 
The  initial  version  of  the  software  was  released  in 
February  2006  at  the  SPIE  Medical  Imaging 
Conference  in  San  Diego,  and  an  updated  version 
was  released  at  the  same  meeting  in  February 
2007.  The  software  and  documentation  is  freely 
available  for  download  at  http://www.igstk.org. 

The  toolkit  was  developed  through  a  collabora¬ 
tion  between  academic  and  industry  partners.  The 
principal  investigator  of  the  project  is  Kevin 
Cleary  at  the  Imaging  Science  and  Information 
Systems  (ISIS)  Center  in  Georgetown  University. 
ISIS  is  a  medical  research  and  development 


institute  that  specializes  in  information  and  imag¬ 
ing  technology  in  healthcare.  The  commercial 
partner  is  Kitware,  a  small  company  specializing 
in  open  source  software.  Kitware  also  provides 
commercial  consulting  services  for  ITK  and  VTK. 
Expertise  in  tracking  systems  was  provided  by 
Atamai  (Ontario,  Canada).  Experts  from  Arizona 
State  University  provide  guidance  in  software 
process  management  and  object-oriented  technol¬ 
ogies.  The  image-guided  research  group  at  Selska- 
pet  for  Industriell  og  Teknisk  Forskning  ved  Norges 
Tekniske  Hoegskole  (SINTEF)  in  Trondheim,  Nor¬ 
way,  has  also  recently  joined  the  project  to  provide 
a  strong  focus  on  end-user  applications. 

The  remainder  of  the  paper  consists  of  five  parts. 
“IMAGE-GUIDED  MEDICAL  PROCEDURES— 
BACKGROUND”  presents  background  informa¬ 
tion  about  image-guided  medical  procedures. 
“IGSTK  SOFTWARE  PROCESS”  describes  the 
software  process  adopted  for  IGSTK  development. 
This  is  followed  by  a  discussion  of  the  architec¬ 
ture  in  “IGSTK  ARCHITECTURE.”  “BUILDING 
AN  IGSTK  COMMUNITY”  outlines  the  strate¬ 
gies  being  followed  to  build  the  IGSTK  user 
community.  Finally,  conclusions  are  presented  in 
“CONCLUSION.” 


IMAGE-GUIDED  MEDICAL 
PROCEDURES- BACKGROUND 

Image-guided  procedures  are  rapidly  replacing 
open  surgical  procedures  in  clinical  practice.  The 
main  driving  forces  behind  the  wide  acceptance  of 
image-guided  procedures  are  technological  advance¬ 
ments  in  medical  imaging,  registration  algorithms, 
visualization  technologies,  and  tracking  systems. 

To  use  medical  images  for  surgical  navigation, 
we  first  need  to  make  them  correspond  with  the 
patient’s  anatomy  in  a  step  called  registration. 
Registration  is  the  technique  of  computing  a  spatial 
transform  that  maps  points  from  one  coordinate 
system  to  another.  When  we  can  accurately  register 
the  image  coordinate  system  with  the  patient 
coordinate  system,  we  can  use  the  images  for 
virtual  guidance  of  surgical  instruments. 

Registration  categories  include  registration  of 
preoperative  images  to  intraoperative  images, 
registration  of  preoperative  images  to  the  patient 
coordinate  system,  and  registration  of  images  from 
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different  modalities.  Registration  techniques 
include  image  intensity-based,  feature-based  (fidu¬ 
cial  landmarks  or  anatomical  landmarks),  surface- 
based,  and  stereotactic  frame-based  techniques4. 
For  example,  x-ray  fluoroscopy,  which  accounts 
for  more  than  90%  of  intraoperative  imaging  for 
guidance,  generates  2D  projection  images.  An 
interventional  radiologist  has  to  mentally  register 
these  2D  projection  images  to  the  3D  patient  body 
to  perform  the  procedure.  This  can  create  ambigu¬ 
ity  and  inaccuracies  in  the  procedure.  However,  by 
registering  a  preoperative  3D  magnetic  resonance 
or  CT  image  to  the  x-ray  fluoroscopic  image,  3D 
image  information  can  be  used  to  guide  the 
procedure. 

Another  essential  component  of  an  image- 
guided  system  is  the  tracking  device.  A  tracking 
system  measures  the  real-time  position  of  surgi¬ 
cal  tools  and  fiducials  attached  to  the  patient’s 
anatomy.  Visual  virtual  feedback  can  be  provid¬ 
ed  by  generating  computer  graphic  images  where 
these  changing  positions  of  the  surgeon’s  tools 
are  superimposed  on  the  preoperative  or  intra¬ 
operative  images.  This  helps  the  surgeon  navi¬ 
gate  inside  the  patient’s  body  by  using  the 
overlaid  images.  The  main  tracking  technologies 
used  in  image-guided  procedures  are  optical  and 
electromagnetic  tracking.  Optical  tracking  sys¬ 
tems  require  an  unobstructed  line  of  sight 
between  the  tracked  sensors  and  tracking  device. 
Electromagnetic  tracking  system  uses  small 
sensor  coils  that  can  be  embedded  in  instruments 
and  does  not  have  the  line  of  sight  limitation. 
Thus,  electromagnetic  tracking  can  be  used  to 
track  instruments  inside  the  body.  However, 
electromagnetic  tracking  is  affected  by  ferro¬ 
magnetic  objects  in  its  working  volume,  and 


care  must  be  taken  to  have  a  relatively  metal- 
free  environment.  An  example  electromagnetic 
tracking  system  is  shown  in  Figure  2. 

To  provide  visual  feedback  to  the  clinician, 
computer  graphic  images  are  generated  where 
registered  information  of  the  tracked  instrument 
and  preoperative  or  intraoperative  image  is  pre¬ 
sented  in  a  meaningful  manner.  Visualization  plays 
a  key  role  in  this  regard.  The  visualization 
application  typically  displays  the  virtual  anatomy 
around  the  location  of  the  surgical  instrument  in 
real-time.  Traditionally,  slice-by-slice  or  multi- 
planar  display  views  are  used.  The  images  can 
also  be  reformatted  to  create  a  view  perpendicular 
to  the  direction  or  plane  of  the  probe.  With  the 
increase  in  the  resolution  of  medical  images, 
surface  and  volume  rendering  techniques  are 
commonly  used  to  display  3D  information  of  the 
surgical  scene.  Surface  rendering  techniques  dis¬ 
play  surface  structures  identified  within  the  image 
(the  surface  could  be  generated  using  contouring 
techniques).  This  may  be  augmented  by  texture 
mapping  in  which  the  original  data  is  pasted  on 
selective  surfaces.  For  volume  rendering,  a  ray¬ 
casting  technique  is  commonly  used  where  the 
displayed  intensity  of  the  image  at  each  point  is  a 
function  of  the  characteristics  of  the  structures 
traversed  by  the  ray.  Advancement  in  these 
visualization  algorithms  and  computing  power 
has  made  interactive  rendering  of  3D  scenes 
possible  in  surgical  procedures. 


IGSTK  SOFTWARE  PROCESS 

IGSTK  is  designed  to  develop  safety-critical 
applications.  This  has  led  us  to  adopt  a  lightweight 


Fig  2.  Aurora  electromagnetic  tracking  system  components,  sensors,  and  measurement  volume.  The  left  picture  shows  (from  left  to 
right )  the  control  unit,  sensor  interface  device,  and  electromagnetic  field  generator.  The  middle  picture  shows  the  sensor  coils  ( red 
objects  in  the  middle  of  the  figure).  The  right  picture  shows  the  measurement  volume  which  is  a  500  mm  cube  starting  50  mm  from  the 
front  of  the  field  generator  (Image  courtesy  of  Northern  Digital,  Inc.). 
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software  process  that  emphasizes  safety  and 
robustness  while  also  supporting  geographically 
separated  developers.  The  adopted  software  pro¬ 
cess  is  philosophically  aligned  with  agile  software 
development  methods  where  lightweight,  iterative, 
incremental,  and  test-driven  development  principles 
are  applied.  The  IGSTK  software  process  has  four 
major  components:  source  code  version  control 
system,  build  and  release  management,  automated 
testing,  and  communication  and  documentation. 

Version  Control  System 

Managing  source  code  versions  is  essential  in  an 
iterative  software  development  process.  IGSTK 
uses  the  concurrent  versioning  system  (CVS).  CVS 
follows  a  client-server  model  where  a  server 
maintains  a  central  repository  for  developers  to 
check  out,  modify,  and  merge  changes  back  to  the 
code  base.  CVS  allows  simultaneous  edits  and 
branching  and  makes  releases  easier.  IGSTK  has 
two  code  repositories:  a  main  CVS  and  a  sandbox. 
The  main  CVS  repository  contains  the  version  of 
the  code  that  has  been  reviewed,  well-tested,  and 
approved  for  integration.  The  code  in  the  main 
CVS  repository  adheres  to  all  the  quality  software 
policies  established  in  IGSTK.  On  the  other  hand, 
the  sandbox  is  a  testbed  for  developers  to 


experiment  on  new  ideas  before  integration  with 
the  rest  of  the  toolkit,  and  it  is  subjected  to  less 
stringent  quality  requirements. 

Build  and  Release  Management  System 

Configuring  and  managing  the  build  process  of 
a  large  project  is  a  challenging  task  in  software 
development.  Large  projects  employ  various  tools 
and  libraries.  Developers  use  different  compilers 
and  development  environments  for  coding.  The 
build  process  should  be  able  to  coordinate  all  these 
tools.  The  task  is  particularly  formidable  for  a 
cross-platform  toolkit  like  IGSTK.  The  cross¬ 
platform  project  configuration  task  includes  select¬ 
ing  an  appropriate  compiler,  finding  required 
packages  and  libraries,  specifying  include  header 
file  and  library  file  paths,  and  selecting  the 
appropriate  compiler  flags  and  options  consistently 
across  platforms. 

The  IGSTK  configuration  and  build  process  is 
controlled  by  Cmake5.  CMake  is  a  cross-platform 
build  system.  CMake  simplifies  the  configuration 
process  by  using  platform-independent  configura¬ 
tion  files  to  generate  native  build  files  such  as 
UNIX  makefiles  and  Visual  Studio  workspaces  for 
the  user-selected  compiler.  CMake  automates  the 
configuration  process  and  makes  it  possible  to 
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Fig  3.  Nightly  dashboard  for  multi-platform  quality  control. 
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develop  a  cross-platform  toolkit  without  too  much 
additional  effort. 

IGSTK  has  official  and  interim  release  cycles. 
The  official  releases  are  generally  planned  once 
a  year.  Official  releases  contain  completed  major 
functionalities  of  the  toolkit  such  as  new 
components.  In  between  official  releases,  the 
team  aims  for  interim  releases  approximately  2 
or  3  months.  Interim  releases  typically  include 
new  features  for  some  components.  Release 
information  is  posted  on  the  IGSTK  Wiki  and 
the  website  (  http://www.igstk.org  ). 

Automated  Testing 

Incremental  development  relies  on  a  continuous 
and  stable  software  code  base.  Testing  is  critical  to 
maintain  the  quality  of  the  software.  Incremental 
development  can  benefit  greatly  from  continuous 
and  automated  testing.  Automated  testing  provides 
instant  feedback  on  the  impact  of  new  additions  to 
the  code  base  or  bug  fixes  on  the  rest  of  the  toolkit. 
Because  IGSTK  is  a  cross-platform  toolkit,  it  is 
tested  on  various  combinations  of  hardware, 
operating  systems,  and  compilers,  and  the  results 
are  reported  back  to  the  developers.  This  is  made 
possible  by  using  the  combination  of  CTest  and 
the  Dart  system.  CTest  is  one  of  the  components  of 
CMake,  and  it  is  intended  to  orchestrate  the 
configuration,  building,  and  testing  of  the  software 
by  gathering  and  submitting  the  results  of 
these  processes  to  a  Dart  server. 


Dart  is  an  open  source  infrastructure  tool 
designed  for  software  quality  control  in  the  context 
of  a  geographically  distributed  development  team. 
In  the  Dart  system,  the  software  that  is  being  tested 
is  run  on  multiple  combinations  of  hardware, 
operating  systems,  and  compilers,  and  the  results 
are  submitted  to  a  central  server,  where  they  are 
made  available  for  review  and  feedback  in  a 
publicly  accessible  web  page.  In  addition  to 
compilation  and  build  results,  code  coverage,  unit 
test,  and  dynamic  analysis  results  are  posted  on 
the  Dart  dashboard.  This  mechanism  ensures  a 
continued  quality  control  of  the  software  during 
the  development  stage.  The  dashboard  accepts 
nightly,  experimental,  and  continuous  build 
results.  Every  night  at  a  specified  time,  the  entire 
IGSTK  toolkit  is  built  on  multiple  machines,  and 
all  the  tests  are  run,  and  the  results  are  posted  on 
the  dashboard. 

Figure  3  shows  a  screenshot  of  a  nightly  IGSTK 
dashboard.  During  the  day,  if  new  changes  are 
committed  to  the  repository,  continuous  builds  are 
submitted  automatically  to  the  dashboard.  In  this 
way,  changes  are  immediately  tested  without 
waiting  for  the  nightly  build,  and  the  developer 
can  easily  trace  problems  that  the  new  changes 
may  have  caused  in  the  rest  of  the  toolkit. 
Experimental  builds  are  used  to  test  locally 
modified  versions  of  the  toolkit.  This  provides 
the  developer  with  feedback  about  the  impact  of 
the  local  change  on  the  rest  of  the  toolkit  before 
committing  it  to  the  repository. 


Fig  4.  State  machine  for  IGSTK  tracker  component  showing  the  four  states  in  black  and  the  transitions  in  blue. 
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Communication  and  Documentation 

Communication  is  essential  for  the  success  of  a 
collaborative  project,  particularly  for  a  geograph¬ 
ically  distributed  development  team.  The  IGSTK 
software  process  was  designed  to  facilitate  effec¬ 
tive  communication  among  developers  and  project 
leads.  Biweekly  teleconferences,  idea  sharing 
using  Wiki  pages,  and  mailing  lists  are  powerful 
and  effective  tools  for  communication.  Technical 
topics  are  discussed  in  great  detail  regularly  during 
the  teleconferences.  Furthermore,  the  dashboard  is 
reviewed  at  each  teleconference  to  maintain  the 
quality  of  the  software. 

Documentation  is  essential  for  the  efficient  and 
continued  use  of  a  toolkit.  IGSTK  follows  the 
literate  programming  philosophy.  Literate  program¬ 
ming  encourages  developers  to  include  human- 
readable  documentation  in  the  source  code.  IGSTK 
uses  the  Doxygen  tool  for  automated  document 
generation  from  the  source  code.  Doxygen  employs 
a  simple  mark  up  language  that  is  embedded  as 
comment  statements  in  the  source  code.  This 
mechanism  has  two  main  benefits.  First,  contrary 
to  traditional  software  development  approaches, 
developers  will  not  have  to  spend  time  after  code 
development  writing  documentation.  Instead  devel¬ 
opers  type  the  documentation  as  they  continue 
implementing  the  code  while  the  ideas  are  still 
fresh  and  while  they  are  in  the  mindset  of  the  code 
logic.  In  this  way,  the  documentation  will  be  ready 
when  code  development  is  finished.  Secondly, 
when  developers  modify  the  code  to  add  a  new 
feature  or  fix  a  big,  they  are  required  to  update  the 
markup  comments  accordingly  so  that  the  docu¬ 
mentation  will  stay  current  with  the  code.  In 


IGSTK,  the  document  generation  process  is  also 
integrated  with  the  nightly  build  system.  Every 
night,  the  documentation  is  generated  and  made 
accessible  from  the  dashboard.  Hence,  users  always 
have  access  to  the  latest  documentation. 

To  summarize  the  software  process,  here  are 
the  ten  best  practices  adopted  by  the  IGSTK 
team6: 

1)  Recognize  that  people  are  the  most  important 
mechanism  available  for  ensuring  high  quality 
software 

2)  Facilitate  constant  communication  among 
developers 

3)  Produce  iterative  releases 

4)  Manage  source  code  from  a  quality  perspective 

5)  Focus  on  100%  code  and  path  coverage  at  the 
component  level 

6)  Emphasize  continuous  builds  and  testing 

7)  Support  the  development  process  with  robust 
tools 

8)  Manage  requirements  iteratively  in  lockstep 
with  code  management 

9)  Focus  on  meeting  exactly  the  current  set  of 
requirements,  and 

10)  Evolve  the  development  process. 


IGSTK  ARCHITECTURE 

IGSTK  supports  development  of  image-guided 
surgery  applications  that  are  classical  examples  of 
safety-critical  applications.  As  a  mechanism  for 
preventing  patient’s  harm,  the  architecture  is  based 
on  the  concept  of  safety-by-design.  In  particular, 
the  IGSTK  team  followed  a  component-based 
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architecture7  and  used  state  machine8  software 
design  methodologies.  Every  IGSTK  component 
has  a  well-defined  set  of  features  that  are  governed 
by  state  machines.  The  state  machine  ensures  that 
the  component  is  always  in  a  valid  state  and  all 
state  transitions  are  valid  and  meaningful.  Figure  4 
shows  the  state  machine  implementation  for  the 
tracker  component  (the  IGSTK  components  will 
be  described  later  in  this  section). 

In  a  component-based  architecture,  the  main 
capabilities  are  broken  down  into  functional 
components  with  well-defined  interfaces  for  inter¬ 
component  communication.  This  type  of  architec¬ 
ture  has  several  benefits  when  developing  a 
reliable  and  robust  toolkit.  Breaking  down  the 
complex  functionalities  into  smaller  pieces  makes 
implementation  manageable  and  makes  it  possible 
to  enforce  higher  standards  of  quality  control.  It  is 
much  easier  to  thoroughly  test  small  components 
with  well-defined  interfaces  than  to  test  medium  or 
large  size  components  with  myriads  of  features 
and  convoluted  interfaces.  Consequently,  com¬ 
plexity  and  implementation  details  are  encapsulat¬ 
ed  in  the  component  level  and  hidden  from  the 
application  developer  or  user.  This  encapsulation 
allows  IGSTK  to  better  manage  the  inherent 
complexity  of  an  image-guided  surgery  system. 
Furthermore,  testing  is  easier  at  the  component 
level  compared  to  traditional  procedural  software 
thereby  increasing  the  quality  of  the  software. 
Application  developers  can  also  use  some  of  the 
components  independently  in  their  application 


without  the  need  to  integrate  the  full  toolkit.  This 
should  increase  the  adoption  of  the  software  and 
subsequently  benefit  the  user  community.  Lastly,  a 
component-based  architecture  is  suitable  for  struc¬ 
tured  toolkit  extension.  This  will  encourage  users 
to  extend  the  toolkit  by  adding  new  components 
and  by  contributing  them  back  to  the  community. 

A  state  machine  is  defined  by  a  set  of  states,  a 
set  of  inputs,  and  a  set  of  directed  transitions  from 
state  to  state.  Each  IGSTK  component  is  subjected 
to  state  machine  control.  Incorporation  of  state 
machines  has  enhanced  the  reliability  of  the  toolkit 
for  the  following  reasons.  State  machines  can 
ensure  that  the  components  have  deterministic 
behavior  at  all  times  and  are  always  in  a  known 
and  error- free  state.  Formal  validation9  of  the 
software  can  be  undertaken  because  inputs,  states, 
and  transitions  are  well  defined  and  are  finite.  This 
framework  is  suitable  for  automated  testing. 
Interaction  between  the  user  and  other  applications 
can  be  explicitly  defined  using  state  machines 
reducing  the  possibility  of  design  and  implemen¬ 
tation  flows  in  application  development.  In  sum¬ 
mary,  state  machines  ensure  safety  and  reliability, 
cleaner  design,  application  programming  interface 
(API)  simplicity,  consistent  integration  pattern,  and 
allow  quality  control.  State  machines  can  also 
prevent  the  misuse  of  components  and  help 
manage  complexity,  traceability,  and  testing. 

IGSTK  components  are  implemented  using  the 
C++  programming  language.  The  component 
implementations  are  mostly  based  on  ITK  and 


Fig  6.  Needle  biopsy  application  system  setup. 
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VTK  classes  subjected  to  a  strict  state  machine 
control.  IGSTK  components  however  do  not 
expose  ITK  or  VTK  classes  in  their  public 
interface.  This  exposure  is  avoided  to  increase  the 
level  of  safety  of  the  components.  Interactions  with 
the  ITK  and  VTK  classes  are  done  only  inside  the 
IGSTK  components  and  only  under  the  supervi¬ 
sion  of  the  state  machine. 

The  IGSTK  architecture  is  shown  in  Figure  5. 
The  main  components  in  IGSTK  are  presented  in 
the  following  list,  and  a  short  description  of  each 
component  is  given: 

•  View  (Display) 

•  Spatial  objects  (geometric  representation) 

•  Spatial  object  representation  (visual 
representation) 

•  Trackers 

•  Readers 


View  (Display) 

Viewers  display  the  graphical  representations  of 
the  renderings  of  surgical  scenes.  Surgeons  perform 
their  task  by  visualizing  the  information  provided  in 
the  viewers.  Furthermore,  the  view  components 
serve  as  a  link  between  the  graphical  user  interface 
library  and  the  rest  of  the  IGSTK  toolkit. 

Spatial  Objects  (Geometrical  Representation) 

Spatial  objects  define  a  common  structure  for 
geometrical  objects  in  IGSTK.  The  spatial  objects 
hold  the  shapes  and  physical  locations  of  objects  in 
the  surgical  environment.  IGSTK  spatial  objects 
encapsulate  ITK  spatial  objects  in  a  restrictive  API. 
Spatial  objects  provided  by  IGSTK  include  objects 
such  as  axes,  boxes,  cones,  cylinders,  images, 
ellipsoids,  meshes,  tubes,  and  vascular  networks. 


Fig  7.  Graphical  user  interface  for  the  needle  biopsy  application. 
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Spatial  Object  Representation  (Visual 
Representation) 

Spatial  object  representations  characterize  the 
graphical  representations  of  the  spatial  object.  The 
graphical  representations  dictate  how  an  object 
should  be  displayed  on  the  screen.  This  will 
include  specifying  color,  opacity,  and  other  ren¬ 
dering  properties. 

Trackers 

The  tracker  component  handles  the  communi¬ 
cation  between  tracking  tools  and  tracking 
devices  to  get  position,  orientation,  and  other 
relevant  information  from  surgical  instruments 
present  in  the  scene.  The  tracking  component 
encapsulates  the  tracking  tool,  tracking  device, 
and  all  static  and  dynamic  information  associat¬ 
ed  with  tracking. 

Readers 

Readers  bring  data  into  the  scene  generation  and 
representation  process.  The  most  important  readers 
in  IGSTK  are  the  Digital  Imaging  and  Communi¬ 
cations  in  Medicine  (DICOM)  image  reader  classes. 
These  classes  are  used  to  read  preoperative  and 
intraoperative  scans  for  surgical  planning  and 
guidance.  Validity  check  logic  is  implemented  in 
these  classes  to  avoid  incorrect  file  reads  and  3D 


volume  generation.  Additional  readers  are  available 
also  for  loading  mesh  and  calibration  data. 

In  addition  to  the  above  main  components, 
IGSTK  has  a  collection  of  infrastructure  and 
service  classes.  The  infrastructure  classes  include 
state  machines,  events,  pulse  generators,  and  real¬ 
time  clock  generator  classes.  Service  classes 
include  loggers,  registration,  and  calibration  clas¬ 
ses.  Loggers  are  useful  for  post-analysis  of  surgical 
procedures  and  recovery  from  a  failure.  Integrating 
loggers  into  applications  streamlines  and  expedites 
the  application  development  process  and  also 
provides  a  suitable  framework  for  verification 
and  validation. 

Another  service  class  in  IGSTK  is  the  registra¬ 
tion  class.  This  class  computes  the  spatial  trans¬ 
formation  between  the  patient  and  the  coordinate 
systems  of  the  multiple  image  data  sets  that  may 
be  present  in  the  scene,  including  preoperative  and 
intraoperative  images.  Finally,  a  calibration  class  is 
available  to  measure  the  most  important  point  of  a 
surgical  instrument  relative  to  the  position  of  the 
tracked  element  on  the  tool  attached  to  the 
instrument. 

Putting  It  All  Together 

To  demonstrate  how  the  components  can  be 
integrated  to  develop  an  application,  we  will 
discuss  an  example  image-guided  needle  biopsy 
application  developed  using  IGSTK. 


Fig  8.  Robot  assisted  needle  placement  phantom  study. 
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Needle  biopsy  is  a  medical  procedure  intended 
>to  take  a  sample  of  tissue  from  a  lump  or  tumor 
or  other  abnormal  growth  in  the  body  with  the 
purpose  of  performing  a  pathological  analysis.  An 
interventional  radiologist  will  insert  a  needle  into 
the  tumor  and  take  a  tissue  sample.  This  is  a 
common  procedure  for  cancer  diagnosis.  It  is 
critical  to  ensure  that  the  needle  reaches  the  target 
that  has  been  identified  from  the  images. 

In  this  application,  the  patient  is  positioned  on  the 
CT  table,  a  CT  image  is  acquired,  and  the  location 
of  the  pathological  tissue  is  identified  on  the 
images.  A  landmark-based  registration  technique  is 
then  used  for  registering  the  image  coordinate 
system  to  the  patient  coordinate  system.  During 
the  biopsy  procedure,  a  computer  graphics-generated 
representation  of  the  surgical  scene  is  presented 
in  the  display  and  is  updated  with  the  current 
position  and  orientation  of  the  needle  as  they  are 


continuously  reported  by  a  tracker.  The  workflow 
of  this  application  is  outlined  below. 

1)  Record  patient  demographic  information 

2)  Acquire  and  transfer  CT  image  to  the  image- 
guided  system 

3)  Identify  landmark  points  in  the  image  using 
the  mouse.  A  minimum  of  three  noncollinear 
landmark  points  are  required. 

4)  Initialize  the  tracking  device 

5)  Identify  the  corresponding  landmarks  in  the 
physical  body  using  the  tracker  pointing  device 

6)  Perform  registration  to  compute  the  transfor¬ 
mation  from  patient  to  preoperative  image 

7)  Identify  entry  point  and  target  position  for  needle 
path  planning 

8)  Start  tracking. 

When  tracking  is  started,  an  updated  position  of 
the  needle  is  displayed  and  overlaid  on  the  CT 


Fig  9.  Graphical  user  interface  for  the  robot  assisted  needle  placement  application. 
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image.  The  radiologist  can  then  manipulate  the 
needle,  watching  the  virtual  image,  until  the  needle 
reaches  the  target.  A  confirming  CT  image  can 
then  be  acquired. 

For  this  example  application,  a  Polaris  Viera 
optical  tracker  (Northern  Digital,  Ontario,  Canada) 
was  used.  Testing  was  performed  using  an  abdom¬ 
inal  phantom  (CIRS  Model  57,  Norfolk,  VA).  The 
application  set  up  is  shown  in  Figure  6.  Figure  7 
shows  the  user  interface,  which  consists  of  a 
control  panel  and  four  standardized  views:  axial, 
sagittal,  coronal,  and  a  3D  view.  The  four  view 
windows  show  CT  images  of  the  abdominal 
phantom  with  the  overlay  of  the  biopsy  needle 
path.  The  green  cylinder  represents  the  needle  as 
tracked  by  the  Polaris  Viera  Tracker.  The  four 
views  automatically  re-slice  and  update  the  images 
to  show  the  needle  tip  position  as  it  moves  in  the 
patient’s  anatomy. 

BUILDING  AN  IGSTK  COMMUNITY 

The  continued  success  and  viability  of  an  open 
source  toolkit  depends  on  strong  user  community 
support.  High  quality  design  and  architectural 
robustness  are  significant  factors  that  drive  the 
adoption  of  open  source  software.  However,  for 
long-term  success  and  continuous  evolution  of  the 
software,  contributions  from  the  user  community 
are  equally  vital.  Beyond  the  first  few  years  of 
development  supported  by  funding  agencies,  the 
lifetime  of  the  toolkit  depends  on  dedicated  users 
and  volunteers  who  care  about  the  toolkit.  Hence, 
as  part  of  the  toolkit  development  effort,  toolkit 
creators  should  treat  building  and  supporting  the 
user  community  as  a  very  important  task.  Without 
a  committed  user  community,  any  open  source 
software  fades  away  after  just  few  years  of 
existence.  With  this  understanding,  the  IGSTK 
team  has  taken  several  key  steps  to  build  a  strong 
IGSTK  community. 

Complete  documentation  encourages  users  to 
evaluate  the  toolkit  and  contribute  modifications 
and  bug  fixes.  Automated  documentation  genera¬ 
tion  techniques  simplify  this  task  in  IGSTK.  Users 
have  access  to  the  latest  API  documentation  from 
the  IGSTK  website  and  from  the  dashboard. 
Furthermore,  to  provide  a  detailed  description  of 
the  toolkit  and  technical  explanation  of  the 
components,  the  IGSTK  team  has  written  and 


published  a  book.10  The  book  is  available  in  a  PDF 
format  for  free  download  at  the  IGSTK  website 
(  http://www.igstk.org  ). 

A  users’  mailing  list  has  also  been  created. 
Questions  posted  in  this  mailing  list  are  promptly 
answered  by  the  developers  of  the  toolkit.  Further¬ 
more,  users  post  bug  reports,  make  feature 
requests,  and  contribute  suggestions  on  how  to 
improve  the  toolkit.  These  suggestions  are  serious¬ 
ly  studied  by  the  development  team,  documented 
in  the  bug  tracker,  and  considered  for  implemen¬ 
tation  as  time  and  resources  permit. 

Demonstrations  of  the  toolkit  is  another  key 
dissemination  effort  undertaken  by  the  group. 
The  needle  biopsy  application  described  above 
was  demonstrated  at  the  SPIE  Medical  Imaging 
Conference  in  2006  and  2007  at  San  Diego,  CA. 
Similarly,  at  the  2007  Society  of  Medical 
Innovation  and  Technology  Conference,  a  robot¬ 
ically  assisted  needle  placement  application  was 
demonstrated.  The  application  guides  the  inser¬ 
tion  of  a  needle  using  the  robot.  Figures  8  and  9 
show  the  robot  set  up  and  a  screenshot  of  the 
graphical  user  interface.  The  application  demon¬ 
strations  have  helped  to  introduce  the  toolkit  to 
the  medical  research  community.  Related  to  this 
effort,  tutorial  sessions  on  IGSTK  have  been 
offered  at  the  SPIE  Medical  Imaging  conference 
as  part  of  the  medical  image  analysis  course  using 
open  source  software. 

What  users  are  allowed  to  do  with  any  open 
source  software  is  defined  by  the  copyright  holder 
in  the  specific  terms  of  the  distribution  license.  For 
this  reason,  the  selection  of  an  appropriate  license 
for  the  toolkit  is  also  essential  in  building  a  strong 
user  community.  The  copyright  of  IGSTK  is  held 
by  the  Insight  Software  Consortium  (  http://www. 
insightsoftwareconsortium.org  ).  IGSTK  is  re¬ 
leased  under  a  Berkeley  Software  Distribution-like 
license,  which  allows  the  use  of  the  software  free 
of  charge  for  academic  and  commercial  applica¬ 
tions.  It  also  allows  users  to  redistribute  the 
software,  modify  it,  and  distribute  the  modifica¬ 
tions  without  requiring  permissions  from  the 
copyright  holders.  An  example  of  the  IGSTK 
commitment  for  building  a  strong  community  is 
the  new  collaboration  that  was  recently  established 
with  the  SINTEF  Health  Research  Center  at 
Norway.  The  center  conducts  cutting  edge  research 
on  developing  advanced  navigation  and  visualiza¬ 
tion  technologies  for  image-guided  surgery.11  The 
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group  is  now  actively  involved  in  the  development 
of  new  components  of  the  IGSTK  toolkit. 


CONCLUSION 

Image-guided  interventions  are  increasingly 
becoming  the  medical  procedure  of  choice 
among  patients  and  clinicians.  They  cause  less 
trauma  to  the  body  and  patients  recover  more 
rapidly  from  these  procedures.  Software  is  a 
critical  component  of  such  systems.  Developing 
reliable  software  for  such  safety-critical  applica¬ 
tions  is  a  challenging  task.  Oftentimes,  research 
institutions  invest  large  amounts  of  resources  to 
build  basic  software  infrastructures  to  develop 
these  applications.  With  the  availability  of  the 
IGSTK  toolkit  that  contains  all  the  components 
needed  to  build  image-guided  applications, 
researchers  will  be  able  to  focus  their  resource 
on  the  main  scientific  problems.  IGSTK  is 
designed  based  on  software  principles  intended 
to  ensure  reliability  and  patient  safety  in  medical 
applications.  The  automated  build  and  test 
management  system  deployed  in  IGSTK  makes 
it  easy  to  accept  and  integrate  contributions  from 
the  user  community  while  maintaining  the  high 
software  quality  standards  established  in  the 
project.  This  is  essential  for  the  continued 
success  and  viability  of  open  source  software. 
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ABSTRACT 

We  have  developed  an  image-guided  navigation  system  using  electromagnetically-tracked  tools,  with  potential 
applications  for  abdominal  procedures  such  as  biopsies,  radiofrequency  ablations,  and  radioactive  seed  placements.  We 
present  the  results  of  two  phantom  studies  using  our  navigation  system  in  a  clinical  environment.  In  the  first  study,  a 
physician  and  medical  resident  performed  a  total  of  1 8  targeting  passes  in  the  abdomen  of  an  anthropomorphic  phantom 
based  solely  upon  image  guidance.  The  distance  between  the  target  and  needle  tip  location  was  measured  based  on 
confirmatory  scans  which  gave  an  average  of  3.56  mm.  In  the  second  study,  three  foam  nodules  were  placed  at  different 
depths  in  a  gelatin  phantom.  Ten  targeting  passes  were  attempted  in  each  of  the  three  depths.  Final  distances  between  the 
target  and  needle  tip  were  measured  which  gave  an  average  of  3.00  mm.  In  addition  to  these  targeting  studies,  we  discuss 
our  refinement  to  the  standard  four-quadrant  image-guided  navigation  user  interface,  based  on  clinician  preferences.  We 
believe  these  refinements  increase  the  usability  of  our  system  while  decreasing  targeting  error. 

Keywords:  abdominal  procedures,  image-guided  therapy,  validation 

1.  INTRODUCTION  AND  BACKGROUND 

Image  guidance  has  become  increasingly  important  in  minimally  invasive  procedures,  in  which  physicians  cannot  directly 
visualize  the  underlying  anatomy.  Previous  image-guided  systems  have  relied  upon  optical  tracking  for  real-time  tool 
location  information.  These  optical  systems  use  a  series  of  cameras  to  triangulate  on  optical  reflectors  placed  on  a  tool 
and  provide  high  spatial  accuracy  at  the  cost  of  requiring  line-of-sight  between  the  cameras  and  reflectors  to  be 
maintained.  Electromagnetic  tracking  systems,  on  the  other  hand,  rely  on  electromagnetic  fields  generated  by  a  field 
generator  which  are  received  by  coils  of  wire  located  inside  the  tool.  Recent  developments  in  electromagnetic  tracking 
technologies,  particularly  the  miniaturization  of  sensor  coils,  have  allowed  researchers  to  create  image-guided  systems 
that  can  track  instruments  inside  the  body. 

Despite  the  benefits  of  electromagnetically-tracked  systems,  they  have  not  been  widely  adopted  for  use  in  image-guided 
applications.  This  may  be  due  to  their  susceptibility  to  electromagnetic  interference  caused  by  ferromagnetic  substances 
placed  within  the  measurement  field.  Several  groups  have  quantified  these  errors  in  laboratory  and  clinical 
environments1,2.  However,  current  versions  of  these  electromagnetic  systems  are  more  resistant  to  small  amounts  of  metal 
in  the  operative  environment.  In  addition,  recent  systems  such  as  the  3D  Guidance  Flat  Transmitter  from  Ascension 
Technology  (Burlington,  VT)  have  also  been  developed  for  operation  in  metal  filled  environments  such  as  an  operating 
room. 

Our  research  group  has  developed  an  image-guided  system  based  upon  an  open  source  software  toolkit3  and  the  Aurora 
electromagnetic  tracking  system  (Northern  Digital  Incorporated,  Waterloo,  Ontario,  Canada).  This  system  provides  real¬ 
time  tool  location  information  overlaid  on  a  pre-procedural  CT,  MRI,  or  other  3D  volumes.  The  paper  presents  the  results 
of  two  phantom  studies  using  this  system  within  a  clinical  environment. 
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2.  MATERIALS  AND  METHODS 


The  first  study  was  designed  to  investigate  targeting  accuracy  in  an  abdominal  phantom  as  performed  by  an  interventional 
radiologist  and  a  medical  resident.  The  second  study  was  designed  to  determine  the  performance  of  our  image-guided 
system  using  targets  placed  at  different  surface  depths  to  simulate  liver  tumors  located  at  different  distances  from  the  skin. 
Both  studies  were  performed  at  the  Georgetown  University  Hospital  in  the  angiography  suite  equipped  with  a  Siemens 
Axiom  dFA  angiography  system  with  a  flat-panel  detector. 

In  the  first  study,  we  used  a  human  torso  phantom  built  by  our  colleagues  at  Johns  Hopkins  University  made  from 
silicone  (TC-5005,  BJB  Enterprises  Inc.,  Tustin,  CA)  and  incorporating  a  synthetic  skeleton4.  We  placed  a  foam  liver 
inside  the  abdominal  cavity  with  several  spherical  (approximately  3  cm  in  diameter)  contrast-enhanced  nodules  to 
simulate  tumors  (see  Figures  1  and  2).  To  gain  access  to  the  foam  liver  with  our  electromagnetically-tracked  tools,  we  cut 
a  2  cm  x  4  cm  rectangular  opening  in  the  chest  of  the  torso  phantom  and  filled  it  with  gelatin  (Knox  Gelatine,  Kraft 
Foods,  Northfield,  IL). 

After  an  initial  rotational  angiography  acquisition  and  3D  reconstruction  (DynaCT),  we  performed  landmark-based 
registration  using  adhesive  skin  fiducials  (Multi-Modality  Markers,  IZI  Medical  Products,  Baltimore,  MD)  placed  on  the 
surface  of  the  torso  phantom.  Care  was  taken  to  not  disturb  the  field  generator  orientation  and  placement  in  relationship  to 
the  torso  after  initial  set  up.  The  average  RMS  registration  error  was  0.82  mm. 


Figure  1:  Anthropomorphic  phantom  and  experimental  set-up  (phantom  constructed  by  URobotics  Laboratory,  Johns 
Hopkins  University) 
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The  physician  or  resident  then  chose  a  skin  entry  location  and  target  location  in  the  phantom  by  clicking  points  on  the 
DynaCT  volume  displayed  on  our  image-guidance  workstation.  Using  electromagnetically-tracked  needles  the  physician 
or  resident  attempted  to  place  the  needle  at  the  target  location  based  solely  upon  image  guidance.  After  needle  placement, 
another  3D  acquisition  was  obtained.  This  volume  was  used  to  measure  the  final  needle  location  in  relation  to  the  selected 
target  point.  Nine  passes  each  were  performed  by  an  interventional  radiologist  and  medical  resident  (for  a  total  of  18 
passes)  within  the  interventional  radiology  suite  at  Georgetown  University  Hospital. 

In  the  second  study,  we  attempted  to  evaluate  the  effect  of  tumor  depth  on  accuracy  for  our  image-guided  system.  We 
created  several  contrast-enhanced  foam  nodules  (Flexfoam,  Smooth-On,  Easton,  PA)  to  simulate  liver  tumors  and 
suspended  them  in  a  gelatin-filled  container  (Knox  Gelatine,  Kraft  Foods,  Northfield,  IL,  see  Figure  3).  Three  foam 
nodules  were  placed  at  three  depths  in  the  gelatin  (approximately  3  cm,  7.5  cm,  and  1 1  cm  from  the  surface).  As  in  the 
first  study,  we  took  an  initial  3D  acquisition  and  used  landmark-based  registration  using  adhesive  fiducials  (see  Figure  4). 
The  average  RMS  registration  error  was  0.76  mm.  We  then  performed  10  passes  in  each  tumor  and  acquired  a  DynaCT 
volume  to  measure  distance  from  target  to  needle  tip  location.  Unfortunately,  we  were  not  able  to  collect  data  from  the 
shallow  tumor  due  to  the  tendency  of  the  needle  to  move  between  final  needle  placement  and  confirmatory  scan.  The 
gelatin  was  not  stiff  enough  to  hold  the  needle  at  that  depth. 


Figure  3:  Picture  of  contrast-enhanced  tumors  and  experimental  set  up  with  tumors  in  gelatin  phantom 


Figure  4:  Experimental  set  up  with  tumors  in  gelatin  phantom  for  study  2 
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In  both  studies,  we  used  a  software  application  based  on  an  open  source  image-guided  surgery  toolkit  (IGSTK)  developed 
by  Georgetown  University  and  partners3.  Instead  of  using  the  standard  four-quadrant  view,  we  worked  closely  with 
several  radiologists  to  create  new  views  to  facilitate  higher  targeting  accuracy  and  greater  ease  of  use  (Figure  5).  A 
patient-centric  perspective  was  chosen  to  closely  mimic  the  clinical  perspective.  In  addition  to  an  axial  view,  we  resliced 
the  dataset  to  create  an  off-axial  and  off-sagittal  viewing  plane  centered  on  the  patient’s  skin  surface  while  containing  the 
electromagnetically-tracked  needle  (Figure  6).  These  planes  followed  the  real-time  position  of  the  tracked  needle  position 
allowing  the  user  to  make  more  realistic  adjustments  to  the  needle  trajectory.  Finally,  we  created  a  view  (bottom  right- 
hand  view  in  Figure  5)  that  is  fixed  to  be  perpendicular  to  the  planned  path.  This  view  included  graphical  indicators  for 
both  the  needle  tip  and  needle  hub  position  and  was  found  to  be  the  most  useful  by  both  the  clinical  and  research  staff  for 
needle  placement  in  the  target.  Each  needle  targeting  pass  in  the  studies  was  started  by  placing  the  needle  tip  (green  ball) 
in  the  center  of  the  target  (red  ball).  Once  the  needle  was  oriented  such  that  the  needle  hub  (yellow  ball)  was  co-located 
with  the  red  and  green  ball,  the  needle  was  advanced  until  the  desired  depth  was  reached.  All  views  were  generated  taking 
into  account  the  position  of  the  screen  relative  to  the  physician  and  patient— all  movements  on  the  screen  matched  the 
actual  movement  of  the  physician. 


Figure  5.  Screenshot  of  user  interface  showing  axial,  off-sagittal,  off-axial,  and  targeting  windows. 
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Figure  6.  Diagrams  of  off-axial  and  off-saggital  views.  Red  lines  denote  needle.  Blue  dashed  lines  indicate  viewing  plane.  Green 
dashed  lines  indicate  axis  of  rotation. 


3.  RESULTS 

For  each  pass  in  study  1,  we  calculated  the  reported  RMS  error  as  displayed  by  the  image  guidance  workstation  after  each 
needle  placement  in  the  target  and  the  actual  RMS  error  between  needle  tip  and  target  position  in  the  confirmatory 
dataset.  The  reported  error  was  calculated  in  the  electromagnetic  coordinate  space,  while  the  actual  error  was  calculated 
by  manually  segmenting  the  needle  tip  position  in  each  confirmatory  scan  and  comparing  it  to  the  selected  target  position 
in  image  coordinate  space.  The  average  over  all  passes  for  actual  error  between  final  needle  tip  position  and  target  was 
3.56  mm  (SD  1.93  mm)  and  the  average  reported  error  was  2.15  mm  (SD  0.78  mm).  All  passes  were  performed  by  the 
clinical  authors  (FB,  JL)  in  the  interventional  suite  at  Georgetown  University  Hospital. 


Study  1 :  Actual  and  Reported  Eitoi 


- ♦ - Actual  Error 

AVG  3.56  mm 

-  -*■  -  Reported  Error 
AVG  2.15  mm 


Figure  7:  Errors  for  Study  1 


All  passes  in  the  second  study  were  performed  in  the  interventional  suite  at  Georgetown  University  Hospital.  We 
surprisingly  measured  a  smaller  needle  tip  placement  error  for  the  deep  target  compared  to  middle  target  and  more 
consistent  accuracy  (denoted  by  the  smaller  standard  deviation)  when  compared  to  study  1 . 
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Study  2:  Actual  and  Reported  Error 


Deep  Target 


Trial 


Middle  Target 


Figure  8:  Errors  for  study  2 


Results  for  Study  2* 

Average  RMS  error  over  20  passes 

3.00  mm  (SD  1.03  mm) 

Average  RMS  error  for  deep  target  (10  passes) 

2.74  mm  (SD  0.79  mm) 

Average  RMS  error  for  middle  target  (10  passes) 

3.26  mm  (SD  1.21  mm) 

*  Shallow  target  was  not  able  to  be  targeted  due  to  the  instability  of  the  gelatin  and  the  tendency  of  the  needle  to  drift  between 
placement  and  confirmatory  scan 


4.  DISCUSSION 

We  present  a  refined  image-guided  navigation  system  based  upon  an  open  source  framework  and  evaluate  the  accuracy  of 
the  system  in  two  phantom  studies.  Previous  studies1,2  have  shown  electromagnetic  interference  in  the  hospital 
environments  to  be  approximately  2-3  mm,  at  best,  and  we  estimate  human  error  in  needle  placement  to  be  1-2  mm,  at 
best.  The  results  of  our  two  phantom  studies  show  an  overall  accuracy  of  3.26  mm  which  include  these  accumulated 
errors  of  electromagnetic  interference,  registration  error,  and  human  error  in  placement  of  the  needle. 

Although  there  is  a  surprising  tendency  in  the  second  study  showing  increased  error  in  the  middle  target,  located  7.5  cm 
from  the  surface,  compared  to  the  deep  target,  located  1 1  cm  from  the  surface,  we  still  see  an  overall  RMS  target  error  of 
3.00  mm.  We  hypothesize  that  the  increase  in  error  for  the  middle  target  was  due  to  the  needle  shift  between  placement 
and  confirmatory  scan.  Further  examinations  with  a  stiffer  substrate  may  be  able  to  explain  this  trend. 

5.  CONCLUSION 

We  have  successfully  developed  an  image-guided  navigation  system  based  upon  an  open  source  software  framework.  By 
using  electromagnetic  tracking,  we  are  able  to  overlay  real-time  instrument  locations  on  pre-procedural  DynaCT  volumes 
thereby  guiding  physicians  to  precisely  place  the  needle  in  desired  locations.  Surveys  among  the  clinical  staff  at 


Page  78 


Georgetown  University  Hospital  show  a  greater  ease  of  use  with  the  refined  four-quadrant  display  developed  for  these 
studies  as  compared  to  previous  iterations  of  the  software  toolkit. 


Our  studies  show,  over  all  38  targeting  passes,  an  average  RMS  distance  from  actual  needle  tip  location  to  planned  needle 
tip  location  of  3.26  mm.  Considering  this  figure  represents  all  accumulated  error  from  the  environment  and  software,  we 
believe  this  could  be  a  promising  platform  for  increasing  the  effectiveness  of  existing  therapies  and  enabling  new 
therapies  currently  limited  by  the  spatial  accuracy  of  non-image-guided  procedures.  We  have  already  begun  to  investigate 
using  this  platform  for  radioactive  seed  placement  in  brachytherapy,  radiofrequency  ablation,  vertebroplasty  and  biopsy 
while  also  investigating  real-time  registration  to  account  for  movement  between  the  time  of  the  pre-procedural  scan  and 
the  procedure  itself. 

Use  of  an  image-guided  planning  system  such  as  the  one  described  above  would  reduce  overall  radiation  exposure  to  the 
patient  and  physician,  especially  compared  to  the  current  advance-and-check  technique  of  percutaneous  access  to  the 
thoraco-abdominal  cavity.  These  systems  could  be  particularly  useful  in  treating  cancers  using  radiofrequency  ablations 
requiring  multiple  ablations  to  completely  ablate  the  tumor5.  Combined  with  computerized  path  planning  based  on  pre¬ 
procedural  imaging,  image  guidance  could  potentially  improve  the  ability  to  treat  cancer  in  these  situations  by  more 
precisely  locating  ablated  regions  and  guiding  physicians  towards  untreated  tumor  tissue. 
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Abstract 

Objective  The  objective  of  this  study  was  to  evaluate  two 
features  of  a  new  rotating  needle  driver  in  a  domestic  swine 
model:  (1)  a  quick  release  safety  mechanism  and  (2)  the 
impact  of  spinning  the  needle  on  the  force  profile. 

Materials  and  methods  The  experiments  were  conducted 
in  a  multi-modality  interventional  suite.  An  initial  CT  scan 
was  obtained  to  determine  the  location  of  the  target,  in  the 
liver  or  lung.  The  robotic  arm  was  positioned  directly  over  the 
marked  skin  entry  point.  Control  parameters  were  set  to  rota¬ 
tion  speeds  of  0, 90,  or  180rpm.  The  breakaway  force  magni¬ 
tude  was  also  preset  to  a  predetermined  force.  The  physician 
used  the  joystick  to  drive  the  needle  towards  the  target  while 
the  system  recorded  needle  insertion  depth  and  forces. 
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Food  and  Drug  Administration,  the  Department  of  Health  and  Human 
Services  or  the  Public  Health  Service. 
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Results  Sixteen  insertions  were  completed  (14  in  liver  and 
2  in  lung)  and  12  released  the  needle  upon  the  desired  set 
force.  The  mean  response  time  of  the  quick  release  mecha¬ 
nism  was  202  zb  39  ms.  Needle  rotation  resulted  in  reduced 
insertion  force. 

Conclusion  The  robot-assisted  needle  insertion  system  was 
shown  to  be  functional  in  a  multimodality  imaging  clinical 
environment  on  a  swine  model.  The  system  has  potential 
future  applications  in  precision  minimally  invasive  proce¬ 
dures  including  biopsy  and  radiofrequency  ablation. 

Keywords  Robotics/instrumentation  •  Safety  • 

Minimally  invasive  •  Rotating  needle  driver 

Introduction 

Minimally  invasive  procedures  have  become  standard  med¬ 
ical  practice  in  many  settings  due  to  numerous  advantages 
such  as  decreased  recovery  time,  cost,  trauma,  and  complica¬ 
tions  in  comparison  to  conventional  surgery.  The  applications 
of  these  procedures  range  over  a  variety  of  specialties 
including  but  not  limited  to  urology,  cardiology,  neurosur¬ 
gery,  orthopedics,  radiation  oncology,  oncology,  and  inter¬ 
ventional  radiology.  Many  of  these  minimally  invasive 
procedures  rely  upon  accurate  insertion  of  a  needle  to  an 
anatomical  target  to  deliver  therapy  or  obtain  tissue  samples 
from  a  pre-determined  specific  location. 

A  few  common  examples  of  needle-based  procedures 
include  biopsy,  radiofrequency  ablation  (RFA),  and  prostate 
brachytherapy.  The  outcome  of  minimally  invasive  proce¬ 
dures  such  as  RFA  depends  on  the  size  and  location  of  the 
target  lesion  and  the  experience  of  the  physician.  For  a  com¬ 
plete  zone  of  ablation,  RFA  requires  the  diameter  of  the  abla¬ 
tion  sphere  to  be  2  cm  greater  than  the  tumor  to  obtain  a  1  cm 
margin  [1].  Prostate  brachytherapy  is  another  needle-based 
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procedure  that  requires  placement  of  radioactive  seeds  at 
pre-planned  locations  around  the  prostate.  Typically  about 
100  radioactive  seeds  are  placed  to  provide  adequate  cov¬ 
erage  of  the  tumor  site.  Currently,  needles  or  electrodes  are 
manually  inserted  into  a  target  by  the  physician,  primarily 
under  image  guidance.  This  requires  a  significant  amount  of 
training,  coordination,  and  2D  to  3D  extrapolation.  Proce¬ 
dures  such  as  these  also  potentially  expose  the  patient  and 
physician  to  significant  radiation.  Inaccuracies  in  needle  or 
electrode  placement  for  these  procedures  and  lack  of  nav¬ 
igation  devices  to  aid  placement  may  result  in  increased 
procedure  time,  morbidity,  inaccurate  diagnosis,  or  tumor 
recurrence  [2]. 

In  recent  years,  robot-assisted  needle  insertion  has 
attracted  attention  because  of  its  capabilities  to  provide  assis¬ 
tance  in  minimally  invasive  procedures.  Computers  and 
robots  have  been  used  in  combination  with  these  procedures 
and  have  demonstrated  enhanced  performance  under  direct 
physician  guidance  simultaneous  with  human  input  as  com¬ 
pared  to  the  more  conventional  method  of  the  physician  alone, 
without  robotic  assistance  [3].  Feasibility  studies  have  been 
presented  by  Solomon  et  al.  [4]  to  develop  a  tumor  abla¬ 
tion  treatment  system  that  utilizes  the  Acubot  robot  system 
for  accurate  applicator  placement  tested  in  patients.  A  roboti¬ 
cally  assisted  needle  insertion  system  for  prostate  brachyther- 
apy  has  also  been  tested  in  phantoms  by  Fichtinger  et  al. 
[5].  The  system  consists  of  a  transrectal  ultrasound  (TRUS) 
and  a  spatially  co-registered  robot.  A  few  other  robotic  sys¬ 
tems  such  as  iGuide  [6,7],  Innomotion  [8],  Pinpoint  from 
Philips  Healthcare  Inc.  [9],  Hata’s  Semi- Active  robot  [10], 
compatible  with  CT  and  MRI,  are  under  active  development 
by  various  researchers  as  well  (see  reviews  [1 1 , 12]  for  a  more 
exhaustive  list).  These  systems  may  provide  a  safer  work¬ 
ing  environment  for  the  physician  by  limiting  the  amount 
of  radiation  exposure.  More  importantly,  they  are  capable 
of  enhancing  outcomes  by  improving  repeatability  and  accu¬ 
racy  which  could  in  turn  help  address  standardization  of  sur¬ 
geon  skill  variability. 

Aside  from  accurate  placement  of  the  needle,  a  major 
safety  concern  is  the  risk  of  tissue  laceration  due  to  respira¬ 
tory,  organ,  or  patient  movement  when  using  a  rigid  robotic 
needle  driver.  When  the  needle  is  inserted  into  the  body, 
patient  movement  due  to  breathing  or  shift  may  cause  lacera¬ 
tion  or  trauma.  Respiration  can  cause  tissue  movement  of  1- 
20  mm  or  more  in  the  area  of  interest  [13],  greatly  hampering 
accuracy.  Currently  physicians  try  to  stop  patient  breathing 
during  needle  insertions  for  up  to  20-25  s  time  intervals  [14] 
and  advance  the  needle  at  a  consistent  phase  of  the  respi¬ 
ratory  cycle.  However,  this  is  primarily  based  on  physician 
judgment  and  is  highly  variable.  The  problem  of  variability  in 
determining  a  consistent  phase  of  respiratory  cycle  becomes 
more  relevant  when  the  needle  is  held  by  a  rigid  device  such 
as  a  robotic  needle  driver. 


For  robot-assisted  needle  insertion  systems  to  be  used  in 
the  clinical  setting,  they  must  overcome  these  safety  issues. 
One  solution  is  to  use  just  a  passive  guide  and  have  the 
physician  insert  the  needle,  as  in  the  Neuromate  [15]  robotic 
system  (Integrated  Surgical  Systems  Inc.,  Davis,  California, 
USA).  At  the  other  end  of  the  spectrum  is  a  fully  active  robotic 
system  such  as  the  Robodoc  [16]  (Integrated  Surgical  Sys¬ 
tem  Inc.)  for  total  hip  replacement.  These  robotic  systems 
have  addressed  safety  concerns,  but  only  in  clinical  settings 
with  predominantly  rigid  structures.  Safety  concerns  for  min¬ 
imally  invasive  robot  assisted  needle  insertions  must  take  into 
account  mobile  soft  tissue  characteristics  and  dynamic  tissue 
deformations  common  in  non-rigid  anatomy. 

Tissue  deformation  and  needle  deflection  are  also  effects 
which  must  be  addressed  for  optimal  robotic  assistance.  Fric¬ 
tional  forces  between  the  needle  shaft  and  tissue  are  one  of  the 
causes  of  tissue  deformation  [17-19].  Some  other  factors  that 
also  affect  tissue  deformation  include  axial  insertion  rate  and 
tip  geometry.  Force  modeling  has  been  applied  to  measure  the 
stiffness,  friction,  and  needle  geometry  effects  during  needle 
insertion  in  soft  tissues  [18,20].  Rotation  of  the  needle  dur¬ 
ing  insertion  has  been  explored  by  multiple  groups  [21,22] 
including  ours,  to  reduce  deformation  and  thereby  decrease 
needle  tip  displacement.  The  effect  of  needle  rotation  dur¬ 
ing  insertion  has  resulted  in  less  tissue  damage,  indentation, 
and  frictional  forces.  High-speed  needle  rotation  in  a  robot 
assisted  prostate  brachytherapy  system  has  also  been  shown 
to  increase  the  accuracy  of  targeting  in  agar  phantoms  [22]. 
Such  rotating  needle  driving  systems  may  allow  for  more 
accurate  needle  insertions. 

The  aim  of  this  paper  is  to  evaluate  the  feasibility  and 
safety  capability  of  a  new  robotic-needle  driving  system  in 
a  swine  model.  The  two  updated  mechanisms:  (1)  a  quick 
release  safety  mechanism  and  (2)  a  rotating  needle  driver 
(RND)  were  evaluated  by  exploring  the  reliability  of  needle 
release  and  impact  of  spin  on  tissue  deformation.  The  time  to 
detect  force  overload  and  needle  release  was  tested  as  a  per¬ 
formance  measure  of  the  quick  release  safety  mechanism. 
To  test  the  impact  of  spin  on  tissue  deformation,  needle 
displacement  and  force  values  were  compared  for  predeter¬ 
mined  spin  speeds.  The  capabilities  of  the  robotic  system 
demonstrated  in  this  preclinical  animal  study  can  potentially 
be  applied  to  a  wide  range  of  other  needle-based,  minimally 
invasive  procedures. 

Materials  and  methods 

The  robotic  system  used  in  these  experiments  is  an  updated 
version  of  the  ‘  AcuBot”  system  built  by  the  Urology  Robotics 
Laboratory  at  Johns  Hopkins  Medical  Institutions  [11,23]. 
The  original  AcuBot  comprises  the  “PAKY”  (Percutaneous 
Access  of  the  KidneY)  needle  driver,  the  “RCM”  (Remote 
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Fig.  1  a  Rotating  Needle 
Driver  (RND),  b  Remote  Center 
of  Motion  (RCM)  orientation 
module,  c/d  Force  sensor, 
e/f  Head  and  Barrel  grippers, 
g  Needle  hub 


Center  of  Motion)  orientation  module,  and  joystick  control. 
A  three  degree  of  freedom  Cartesian  stage,  passive  position¬ 
ing  S-arm,  and  “bridge  frame”  enable  the  system  to  achieve 
a  compact  and  flexible  design  for  interventions  at  multiple 
points  along  the  body. 

Prior  experiences  using  the  AcuBot  system  in  clinical  tri¬ 
als  for  spinal  nerve  blocks  showed  the  need  for  some  system 
enhancements  [24].  The  main  focus  of  these  enhancements 
was  a  complete  redesign  of  the  needle  driver.  Three  new 
components  were  added  to  the  needle  driver:  (1)  a  mecha¬ 
nism  to  spin  the  needle,  (2)  force  sensors,  and  (3)  a  needle 
release  mechanism.  The  updated  system  is  referred  to  as  the 
“AcuBotl  V2-RND”  and  shown  in  Fig.  1. 

The  compact  RND  holds  the  needle  from  two  points  for 
enhanced  support  and  accurate  insertions.  This  design  pre¬ 
vents  buckling  of  long  thin  needles.  Moreover,  it  is  capable 
of  spinning  the  needle  during  insertion  in  either  direction. 
Rotation  of  the  needle  may  help  to  reduce  the  resistive  forces 
by  the  “drilling  effect.”  Needle  rotation  may  also  be  favorable 
by  changing  a  higher  static  friction  between  tissue  and  needle 
to  a  lower  kinetic  friction  thus  reducing  insertion  forces. 

One  unique  feature  of  the  needle  driver  is  the  built-in 
force  sensing  capability.  There  is  a  custom  force  sensor  built 
into  the  lower  arm  that  holds  the  nozzle  (Fig.  1).  As  can  be 
seen  in  Fig.  2  the  mechanical  structure  of  the  lower  arm  has 
been  machined  so  that  it  will  deflect  very  slightly  when  any 
force  is  applied  to  the  nozzle  holding  the  needle.  Strain  gages 
have  been  mounted  on  the  lower  arm  to  sense  this  deflection. 
Through  a  calibration  process  and  calibration  matrix,  the  out¬ 
put  of  these  strain  gages  is  converted  into  three  orthogonal 


forces  at  the  nozzle  tip.  Therefore,  this  force  sensing  capa¬ 
bility  measures  the  interaction  between  the  needle  shaft  and 
the  surrounding  tissue. 

The  needle  driver  can  also  measure  the  axial  force  along 
the  needle.  This  force  is  calculated  by  sensing  the  torque 
applied  to  the  mechanism  driving  the  upper  arm  of  the  nee¬ 
dle.  This  torque  is  then  converted  to  a  force  along  the  needle 
direction  by  accounting  for  the  kinematics  of  the  drive  train 
mechanism.  There  is  some  noise  from  friction  in  this  mea¬ 
surement,  but  by  pressing  manually  along  the  direction  of  the 
needle  and  observing  the  reported  force,  it  has  been  observed 
that  this  capability  provides  a  reasonable  representation  of  the 
actual  force. 

A  safety  mechanism  is  also  built  into  the  needle  driver 
to  release  the  needle  which  can  be  triggered  manually  or 
upon  a  desired  force  level  measured  by  the  force  sensors. 
The  needle  is  quickly  released  from  the  two  grippers,  one  at 
the  head  of  the  needle  and  one  close  to  the  skin  as  shown 
in  Fig.  1.  The  gripper  at  the  head  of  the  needle  controls  the 
spin  and  inserts  the  needle  while  the  second  gripper,  close 
to  the  skin,  guides  its  direction.  Both  grippers  were  fabri¬ 
cated  from  Delrin  plastic  and  were  designed  to  include  two 
finger-like  arms  which  clip  together  to  hold  the  needle  and 
swing  aside  during  release  as  shown  in  Fig.  1.  These  grip¬ 
pers  are  low  cost  and  can  be  easily  manufactured  to  accom¬ 
modate  standard  needle  sizes.  They  can  be  sterilized  and  are 
disposable. 

Experiments  to  test  the  AcuBotl  V2-RND  system  were 
performed  on  domestic  swine  in  a  multi-modality  interven¬ 
tional  suite,  specifically  designed  for  translational  and 
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Fig.  2  Custom  force  sensor 
built  into  the  lower  arm  and 
based  on  strain  gage  readings  as 
shown  in  the  right  hand  figure. 
The  darker  colors  indicate 
higher  strains 


Fig.  3  a  AcuBotl  V2-RND 
mounted  on  the  control  cabinet, 
b  Custom  “bridge  frame”, 
c  Needle  driving  unit,  d  Needle 
driver,  e  Computer  system 
screen, /Joystick  control 


pre-clinical  evaluation  of  image-guided  devices.  All  animal 
care  and  use  procedures  were  approved  by  the  Institu¬ 
tional  Animal  Care  and  Use  Committee.  A  domestic  swine 
(100-125  lbs)  was  initially  sedated  with  a  mixture  of  keta¬ 
mine,  xylazine,  telazol,  and  butorphanol,  then  intubated  and 
maintained  under  general  anesthesia  with  isofluorane.  Once 
the  animal  was  placed  supine  on  the  CT  table,  the  AcuBotl 
system  was  mounted  onto  the  table  using  a  custom  designed 
mount  and  “bridge  frame”,  shown  in  Fig.  3a.  Figure  3b  shows 
the  animal  setup  with  the  system  positioned  over  the  ani¬ 
mal  on  the  CT  table.  A  preliminary  CT  scan  was  obtained  to 
determine  the  location  of  the  target  in  the  liver.  For  each  inser¬ 
tion,  a  different  target  and  skin  entry  point  were  determined 
by  the  physician.  The  physician  then  marked  the  skin  entry 
point  on  the  pig.  After  proper  securing  of  an  18  gauge  15  cm 
Diamond  GREENE  tip  (COOK  Biotech)  needle  within  the 
needle  driver,  the  robotic  arm  was  positioned  directly  over 
the  marked  skin  entry  point. 


The  control  parameters  were  then  set  to  one  of  the  three 
different  rotation  speeds:  0,  90,  or  180rpm.  Other  literature 
has  published  work  with  max  spin  values  up  to  2,000  rpm  [25] 
however,  for  this  system  the  maximum  spin  is  1 80  rpm.  Thus, 
we  chose  this  to  be  our  maximum  rotation  speed  and  an  inter¬ 
mediate  rotation  speed  of  90  rpm  was  also  selected.  At  each 
of  these  speeds  6,  7,  and  3  needle  insertions  were  performed 
respectively.  A  breakaway  force  magnitude  was  also  selected 
so  that  needle  release  would  always  occur  due  to  respiratory 
motion.  This  value  was  enough  to  accommodate  respiratory 
forces  but  then  releasing  the  needle  at  greater  forces  might 
cause  laceration.  The  physician  also  created  a  small  incision 
at  the  marked  point  to  facilitate  needle  advancement.  This  is 
not  a  standard  practice  in  the  clinical  setting.  However,  due 
to  the  thickness  of  porcine  skin  compared  to  human  skin,  this 
was  necessary  to  allow  the  needle  to  be  inserted.  The  pig’s 
breath  was  held  at  the  end  of  inspiration  while  the  physi¬ 
cian  used  the  joystick  to  drive  the  needle  towards  the  target. 
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Fig.  4  A  sequence  of  images 
capture  the  insertion  of  the 
needle  using  the  rotating  needle 
driver,  and  the  subsequent 
smooth,  quick  release  of  the 
needle  at  the  end  of  insertion. 
1-3  display  of  needle  insertion, 
4-5  beginning  and  end  time  of 
needle  breakaway,  6  end  of 
needle  release 


The  control  software  of  the  robot  then  recorded  the  needle 
insertion  depth  and  forces  in  the  X,  Y,  and  Z  directions  for 
analysis  as  described  later. 

Figure  4  shows  a  typical  smooth,  continuous  insertion  of 
the  needle  and  its  release  after  reaching  the  target,  triggered 
by  the  sensed  forces.  Sequences  1-3  show  the  needle  being 
inserted  into  the  pig,  while  sequences  4  and  5  display  the 
beginning  and  end  time  of  complete  needle  release  from  the 
system.  Alternatively,  the  release  could  also  be  manually  trig¬ 
gered  by  the  radiologist  once  the  insertion  is  completed. 

Results 

A  total  of  16  insertions,  14  in  liver  and  2  in  lung,  were  per¬ 
formed.  The  two  lung  insertions  were  performed  to  see  if 
there  was  a  difference  with  the  needle  driver  performance 
between  lung  and  liver.  There  was  no  significant  difference 
in  the  two  lung  trials  (insertions  15  and  16  in  Fig.  5b)  and 
the  liver  trials.  Nonetheless,  of  these  insertions  12  released 
completely  upon  the  desired  set  force  and  four  failed.  Two  of 
the  four  failures  were  released  before  reaching  proper  depth 
during  insertion  and  two  did  not  release  at  all.  Needle  release 
prior  to  achieving  proper  depth  was  due  to  the  needle  interac¬ 
tion  with  more  compact  tissue,  specifically  muscle.  Though 
the  experiments  were  not  explicitly  designed  to  evaluate  sen¬ 


sitivity  of  the  force  measures,  these  two  failures  suggest  that 
the  quick  release  mechanism  should  be  adjusted  to  account 
for  tissue  variation  based  on  CT  image  data.  Other  models  to 
determine  force  profiles  for  this  safety  mechanism  are  being 
investigated.  The  other  two  failures  that  did  not  release  were 
primarily  due  to  excessive  use  and  deterioration  of  the  Delrin 
grippers.  However,  in  the  clinical  setting,  this  would  not  be 
an  issue  because  such  low  cost,  easily  manufactured  parts 
can  be  disposed  of  after  a  certain  number  of  insertions. 

The  response  time  of  the  quick  release  mechanism  is  also  a 
crucial  factor  in  ensuring  safety  during  needle  insertions.  To 
determine  the  response  time,  we  analyzed  the  norm  of  lateral 
forces  (both  x  and  y  directions).  The  number  of  clock  ticks 
between  the  controller  detecting  the  pre-determined  release 
threshold  and  the  norm  of  the  nozzle  force  falling  below 
0.1  N  was  recorded.  The  periodicity  of  the  clock  was  30  ms. 
Figure  5a  shows  a  typical  profile  of  the  recorded  nozzle 
force  annotated  with  the  start  of  needle  release  and  instance 
when  the  needle  is  floating  freely.  Figure  5b  displays  the  time 
between  detection  of  a  predetermined  breakaway  force  and 
needle  release.  The  mean  for  this  measurement  was  202  ± 
39  ms. 

A  plot  of  the  force  versus  needle  displacement  curve  and 
average  values  of  area  under  this  curve  with  respect  to  needle 
rotation  speed  is  shown  in  Fig.  6.  Also  shown  in  Fig.  6b  are 
the  maximum  and  minimum  values  for  each  of  the  spin  rates. 
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Fig.  5  a  A  typical  profile  of 
nozzle  force  recorded  by  the 
strain  gauges,  b  Time  between 
breakaway  force  and  needle 
release 


Fig.  6  a  Force  versus  needle 
displacement  curve,  b  Average 
areas  under  the  Force  versus 
Needle  displacement  curves  of 
12  successful  insertions  for 
three  different  spin  rates.  Thin 
bars  represent  the  Range  of 
values  obtained  for  this  metric 


(b) 


200 


^Max 


0  90 

Needle  Spin  RPM 


As  seen,  there  is  a  marked  reduction  in  the  variability  of  this 
metric  with  rotation.  This  data  shows  the  potential  for  needle 
rotation  to  reduce  insertion  force. 

Discussion  and  conclusions 

The  effects  of  needle  rotation  and  orientation  reversal  at  half 
of  the  insertion  depth  have  also  been  shown  to  increase  the 
accuracy  of  targeting  by  other  researchers  [17,22].  This  gain 
in  accuracy  has  been  shown  to  be  due  to  reduced  frictional 
forces  between  the  needle  shaft  and  the  tissue.  However,  in 
in  vivo  animal  models  difficulties  arise  when  measuring  the 
contributions  obtained  from  needle  rotation,  due  to  factors 
such  as  respiratory  motion  and  organ  movement.  The  effect 
of  needle  rotation  is  also  likely  to  be  related  to  tissue  type, 
needle  beveling  and  tip  shape.  Furthermore,  the  inhomoge- 
nous  mechanical  properties  of  soft  tissue  make  direct  mea¬ 
surement  of  these  frictional  forces  between  the  needle  shaft 
and  tissue  complex  and  noisy  as  seen  by  the  nozzle  force  sen¬ 
sor.  Thus,  to  evaluate  the  effect  of  needle  rotation  we  looked 
at  the  area  under  the  force  profile  with  respect  to  the  nee¬ 
dle  insertion  depth.  The  intuition  behind  this  metric  is  that 
it  can  act  as  a  proxy  for  the  work  done  by  the  needle  driver 
in  inserting  the  needle.  This  measurement  was  collected  at 
different  points  in  the  swine  liver  and  values  were  averaged 
to  reduce  the  effect  of  tissue  inhomogeneity. 


It  has  been  suggested  [21]  that  continuous  rotation  of  the 
needle  might  cause  more  tissue  damage  as  a  result  of  defects 
in  the  shape  of  the  needle.  Our  observations  have  been  in  line 
with  this  suggestion  at  high  speeds,  though  not  necessarily 
at  lower  speeds.  However,  our  design  provides  two  contact 
struts  or  points  of  support  for  the  needle,  one  at  the  head 
and  the  second  close  to  the  skin  entry  point.  This  reduces 
the  chance  of  buckling  or  deformation  of  the  needle  due  to 
insertion  forces. 

In  conclusion,  a  novel  robotic  needle  driver  was  presented 
as  applied  to  an  in  vivo  swine  model.  The  system  was  shown 
to  be  functional  in  a  multimodality  imaging  clinical  envi¬ 
ronment  and  provided  a  compatible  workflow.  The  quick 
release  safety  mechanism  of  the  system  was  demonstrated 
as  a  safety  tool  which  may  prevent  tissue  laceration  and 
trauma,  and  could  be  a  solution  to  a  major  hurdle  facing  active 
needle  drivers  for  standard  clinical  practice  in  needle-based 
procedures.  Safety  mechanisms  such  as  these  are  necessary 
for  wider  acceptance  of  robotic  assistance  in  needle-based 
procedures.  The  response  time  for  the  quick  release  safety 
mechanism  was  sufficient  for  this  proof  of  concept  study. 
The  RND  may  also  provide  an  advantage  by  reducing  tissue 
deformation  and  increasing  the  accuracy  of  needle  targeting. 
Although  the  focus  of  this  study  was  based  on  biopsy  proce¬ 
dures,  this  robotic  system  can  potentially  be  applied  to  a  wide 
range  of  other  needle-based  minimally  invasive  procedures. 
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ABSTRACT 

When  choosing  an  Electromagnetic  Tracking  System  (EMTS)  for  image-guided  procedures,  it  is  desirable  for  the 
system  to  be  usable  for  different  procedures  and  environments.  Several  factors  influence  this  choice.  To  date, 
the  only  factors  that  have  been  studied  extensively,  are  the  accuracy  and  the  susceptibility  of  electromagnetic 
tracking  systems  to  distortions  caused  by  ferromagnetic  materials.  In  this  paper  we  provide  a  holistic  overview 
of  the  factors  that  should  be  taken  into  account  when  choosing  an  EMTS.  These  factors  include:  the  system’s 
refresh  rate,  the  number  of  sensors  that  need  to  be  tracked,  the  size  of  the  navigated  region,  system  interaction 
with  the  environment,  can  the  sensors  be  embedded  into  the  tools  and  provide  the  desired  transformation 
data,  and  tracking  accuracy  and  robustness.  We  evaluate  the  Aurora  EMTS  (Northern  Digital  Inc.,  Waterloo, 
Ontario,  Canada)  and  the  3D  Guidance  EMTS  with  the  flat-panel  and  the  short-range  field  generators  (Ascension 
Technology  Corp.,  Burlington,  Vermont,  USA)  in  three  clinical  environments.  We  show  that  these  systems  are 
applicable  to  specific  procedures  or  in  specific  environments,  but  that,  no  single  system  is  currently  optimal  for 
all  environments  and  procedures  we  evaluated. 

Keywords:  image-guided  therapy,  electromagnetic  tracking,  accuracy  analysis,  usability  study 

1.  INTRODUCTION 

To  date  Electromagnetic  Tracking  Systems  (EMTS)  evaluation  has  only  focused  on  system  accuracy  and  stability 
in  the  clinical  environment.1  3  In  this  paper  we  identify  a  comprehensive  set  of  factors  that  influence  the 
applicability  of  an  EMTS  in  the  clinical  environment.  We  then  evaluate  two  commercially  available  systems 
based  on  these  factors  as  they  pertain  to  several  medical  procedures. 

We  have  previously  developed  image-guidance  systems  based  on  electromagnetic  tracking  for  a  variety  of 
procedures.  These  include  Radio  Frequency  Ablation  (RFA)  of  liver  tumors,4  creation  of  a  Transjugular  In- 
trahepatic  Portosystemic  Shunt  (TIPS),5  carotid  stent  deployment,6  vertebroplasty,7  needle  biopsies  of  liver 
lesions,8  and  transbronchial  biopsies.9 

These  procedures  are  performed  in  three  different  environments:  an  interventional  radiology  suite,  a  CT 
suite,  and  a  pulmonology  suite  (Figure  1).  Our  primary  environment  is  the  interventional  radiology  suite.  It 
houses  a  floor  mounted  C-arm  based  cone-beam  CT  system,  the  Siemens  Axiom  Artis  dFA,  which  provides  both 
volumetric  images  and  projection  images.  The  first  four  procedures,  RFA,  TIPS,  carotid  stent  deployment,  and 
vertebroplasty,  are  performed  in  this  room.  They  utilize  tracked  needles,  catheters,  and  vertebroplasty  trochars. 
The  liver  lesion  biopsies  are  performed  in  the  CT  suite,  and  utilize  needles.  This  suite  houses  a  Siemens  Somatom 
Volume  Zoom  CT  machine  that  provides  volumetric  data  and  also  real  time  single  slice  imaging,  CT-fluoroscopy. 
Finally,  the  transbronchial  biopsies  are  performed  in  the  pulmonology  suite,  using  biopsy  forceps  inserted  through 
the  flexible  bronchoscope’s  working  channel.  The  bronchoscope  we  use  in  this  suite  is  the  Pentax  EB-1530T2. 
Figure  2  shows  the  various  tools  used  in  these  procedures. 
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Figure  1.  Setup  of  electromagnetic  tracking  systems  in  the  interventional  environments  in  which  we  assess  their  usability 
(a,b)  interventional  radiology  suite,  (c,d)  CT  suite,  and  (e,f)  pulmonology  suite.  Left  column  shows  the  setup  with  the 
Aurora  system.  3D  Guidance  short-range  field  generator  setup  is  similar.  Right  column  shows  the  setup  with  the  3D 
Guidance  flat-panel  field  generator. 
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Figure  2.  Tools  used  in  the  clinical  procedures  we  study.  From  left  to  right,  needle,  catheter,  vertebroplasty  trochar, 
biopsy  forceps. 


Ideally,  we  would  like  to  have  a  single  EMTS  that  is  applicable  across  procedures  and  environments.  Based 
on  the  set  of  procedures  enumerated  above  we  define  the  following  requirements  from  this  ideal  EMTS: 

1.  Speed:  refresh  rate  of  100Hz  with  a  latency  of  less  than  1ms,  regardless  of  the  number  of  deployed  sensors. 

2.  Concurrency:  tracks  up  to  30  sensors  concurrently  (note  that  a  flexible  tool  may  contain  multiple  sensors). 

3.  Working  volume:  has  an  effective  work  volume  of  53m  (room  sized). 

4.  Obtrusiveness:  sensors  are  wireless  and  can  function  for  several  hours,  all  hardware  components  can  be 
positioned  so  that  they  do  not  restrict  the  physical  access  to  the  patient,  and  the  system  does  not  have 
any  effect  on  other  devices  used  during  the  procedure. 

5.  Completeness:  sensors  are  small  enough  to  embed  in  any  tool  and  provide  all  six  degrees  of  freedom  (6DOF 
sensors) . 

6.  Accuracy:  resolution  less  than  1mm  and  0.1°. 

7.  Robustness:  not  affected  by  the  environment  (light,  sound,  ferromagnetic  materials,  etc.). 

Having  established  the  requirements  from  the  ideal  EMTS  we  are  now  ready  to  evaluate  existing  EMTS, 
based  on  the  way  they  address  these  requirements. 

As  far  as  we  are  aware,  there  are  only  three  vendors  for  stand-alone  EMTS,  Northern  Digital  Inc.  (Waterloo, 
Ontario,  Canada),10  Ascension  Technology  Corp.  (Milton,  Vermont,  USA),11  and  Polhemus  (Burlington, 
Vermont,  USA).12  In  this  study  we  evaluated  the  Aurora  (Northern  Digital  Inc.),  and  3D  Guidance  (Ascension 
Technology  Corp.)  systems.  We  did  not  evaluate  any  of  the  tracking  systems  from  Polhemus,  as  the  sensor 
size  used  by  these  systems  is  on  the  scale  of  several  centimeters  which  precludes  embedding  them  in  any  of  the 
medical  devices  which  we  need  to  track. 

2.  MATERIALS  AND  METHODS 

In  this  work  we  assess  the  Aurora  and  3D  Guidance  EMTS.  Both  EMTS  consist  of  three  basic  components,  the 
field  generator,  a  system  control  unit  that  interfaces  with  a  PC,  and  tracked  sensor  coils  and  their  respective 
interface  to  the  system  control  unit.  Note  that  the  3D  Guidance  system  is  evaluated  in  two  configurations,  using 
the  flat-panel  and  the  short-range  field  generators.  Figure  3  shows  the  system  components  for  both  EMTS  as 
used  in  this  study. 
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Figure  3.  Electromagnetic  tracking  systems  and  their  components,  as  used  in  this  study,  (a)  Aurora  and  (b)  3D  Guidance. 


From  our  prior  experience  with  assessing  the  accuracy  of  electromagnetic  tracking  systems3  we  have  observed 
variable  performance.  We  thus  believe  that,  while  electromagnetic  tracking  is  a  viable  option  for  tracking  in  the 
clinical  environment  its  applicability  should  be  evaluated  per  environment,  and  even  more  specifically  on  a  per 
procedure  basis. 

We  thus  evaluate  each  requirement  with  respect  to  the  specific  environments  or  procedures  where  relevant. 

2.1.  Speed 

The  maximal  refresh  rate  for  the  Aurora  system  according  to  manufacturer  specifications  is  40Hz.  We  empirically 
evaluated  this  by  acquiring  time  stamped  measurements  using  a  single  five  degree  of  freedom  (5DOF)  sensor. 
Data  was  acquired  with  a  program  developed  in  house  using  the  system’s  API.  The  refresh  rate  we  obtained  was 
39.86Hz  which  concurs  with  the  manufacturer’s  specifications.  When  an  additional  5DOF  sensor  was  attached 
the  refresh  rate  dropped  to  20-25Hz.  Attaching  additional  sensors  after  this  initial  decrease  in  refresh  rate  did 
not  have  any  effect.  For  our  purposes  this  refresh  rate  is  just  sufficient  as  our  display  is  updated  at  25-30Hz. 
System  latency  was  not  assessed  quantitatively.  We  have  qualitatively  observed  that  it  was  not  an  issue  in  any 
of  our  image-guidance  applications. 

The  maximal  refresh  rate  for  the  3D  Guidance  system  according  to  manufacturer  specifications  is  160Hz  with 
the  flat-panel  field  generator  and  375Hz  with  the  short-range  field  generator.  Interestingly,  the  factory  default 
refresh  rate,  set  for  optimal  accuracy,  for  the  flat-panel  field  generator  is  about  40Hz  and  for  the  short-range 
field  generator  about  68Hz.  We  empirically  evaluated  the  refresh  rates  both  for  the  flat-panel  and  short-range 
field  generators  by  acquiring  time  stamped  measurements  using  six  degree  of  freedom  (6DOF)  sensors.  Data  was 
acquired  using  a  custom  program  from  Ascension.  The  refresh  rate  we  obtained  for  the  flat-panel  field  generator 
was  approximately  160Hz  irrespective  of  the  number  of  tracked  sensors.  The  configuration  with  the  short-range 
field  generator  was  also  unaffected  by  the  number  of  tracked  sensors,  with  an  acquisition  rate  of  190Hz. 

2.2.  Concurrency 

The  Aurora  system  supports  tracking  of  up  to  eight  5DOF  sensors  or  four  6DOF  sensors.  The  3D  guidance 
system  supports  tracking  of  up  to  12  5DOF  sensors  and  eight  6DOF  sensors.  For  all  of  our  applications  we  have 
found  these  numbers  to  be  sufficient,  as  less  than  eight  tracked  tools  are  used  at  the  same  time. 

2.3.  Working  volume 

The  Aurora  system  provides  a  working  volume  of  500  x  500  x  500mm.  The  3D  guidance  system’s  working 
volume  is  dependent  upon  the  electromagnetic  field  generator.  With  the  flat-panel  field  generator  the  work¬ 
ing  volume  is  similar  to  the  Aurora  work  volume.  With  the  short-range  field  generator  the  work  volume  is 
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1000  x  1000  x  1000mm  but  manufacturer  specifications  state  that  optimal  accuracy  is  limited  to  a  sub- volume 
of  approximately  200  x  200  x  200mm. 

The  work  volume  provided  by  the  3D  Guidance  short-range  field  generator  will  require  special  consideration 
when  positioning  it  relative  to  the  patient.  For  all  of  our  procedures,  except  for  carotid  stent  deployment  a 
work  volume  of  500  x  500  x  500mm  is  sufficient.  For  carotid  stent  deployment  the  initial  part  of  the  navigation, 
starting  at  the  femoral  artery,  was  performed  using  the  current  clinical  approach,  and  once  the  tracked  volume 
was  reached  the  physician  was  able  to  use  our  navigation  system. 

2.4.  Obtrusiveness 

As  this  requirement  consists  of  several  sub-categories  it  can  be  partially  assessed  independently  from  the  pro¬ 
cedure  and  environment.  Both  the  Aurora  and  3D  Guidance  systems  are  wired.  That  is,  the  tracked  tools  are 
connected  via  wires  to  the  control  unit.  While  this  does  not  preclude  any  of  the  procedures  we  are  investigating 
it  does  make  them  more  cumbersome.  More  importantly,  these  additional  wires  require  caution  on  the  part  of 
the  medical  staff  when  moving  around  the  patient.  If  the  procedure  also  involves  moving  imaging  apparatus  such 
as  the  rotating  C-arm  in  the  case  of  cone-beam  CT,  one  must  exercise  caution  in  placing  the  wires  as  they  are 
not  an  obstacle  during  navigation  but  they  may  be  in  the  path  of  the  rotating  arm. 

Interventional  radiology  suite 

In  all  procedures  performed  in  the  interventional  radiology  suite  the  Aurora  EMTS  is  placed  such  that  the 
systems  field  generator  is  mounted  on  a  passive  mechanical  arm  positioned  approximately  150mm  from  the 
patient  (Figure  1(a)).  This  restricts  physical  access  from  certain  directions  but  has  not  been  a  limiting  factor 
in  any  of  the  procedures.  The  3D  Guidance  system  with  the  short-range  field  generator  is  positioned  similarly. 
It  is  slightly  less  intrusive  than  the  Aurora  system  due  to  the  smaller  form  factor  of  the  field  generator.  The 
3D  Guidance  system  with  the  flat-panel  field  generator  does  not  change  the  physical  access  to  the  patient  from 
current  clinical  practice,  as  the  field  generator  is  placed  underneath  the  patient  (Figure  1(b)). 

In  the  RFA,  TIPS  and  stent  deployment  procedures  preoperative  CT  data  is  used  for  navigation  and  C-arm 
fluoroscopy  is  potentially  used  for  intraoperative  imaging,  validating  the  information  presented  by  the  navigation 
system.  The  Aurora’s  field  generator  precludes  imaging  from  certain  C-arm  poses.  This  is  not  a  limiting  factor 
as  long  as  imaging  and  tracking  are  not  required  simultaneously,  as  the  field  generator  can  be  easily  moved  out 
of  the  way.  For  the  vertebroplasty  procedure  preoperative  cone-beam  CT  data  acquired  with  the  C-arm  is  used 
for  navigation.  This  requires  additional  care  in  positioning  the  field  generator  so  that  it  is  not  in  the  path  of  the 
rotating  C-arm.  Again,  this  was  found  to  be  a  minor  inconvenience. 

The  3D  Guidance  system  with  the  short-range  field  generator  is  similar  to  the  Aurora  system.  When  config¬ 
ured  with  the  flat-panel  field  generator  we  have  found  that  because  the  field  generator  cannot  be  easily  moved 
in  and  out  of  tracking  position  image  quality  is  degraded.  This  was  assessed  by  imaging  an  interventional  3D 
Abdominal  Phantom  (CIRS,  Norfolk  VA,  USA).  This  phantom  is  composed  of  materials  that  mimic  the  X-ray 
attenuation  of  human  tissue.  We  acquired  both  2D  projection  images  and  a  C-arm  cone-beam  CT  reconstruction, 
with  and  without  the  flat-panel  field  generator  in  ] 

CT  suite 

In  the  needle  biopsy  procedures  performed  in  the  CT  suite  the  Aurora  EMTS  is  placed  such  that  the  system’s 
field  generator  is  mounted  on  a  passive  mechanical  arm  positioned  approximately  150mm  from  the  patient 
(Figure  1(c)).  This  restricts  physical  access  to  one  side  of  the  patient  but  has  not  been  a  limiting  factor  in  this 
procedure  as  the  mechanical  arm  and  field  generator  are  on  the  opposite  side  of  the  CT  bed  from  the  physician. 
As  was  the  case  in  the  interventional  radiology  suite,  the  3D  Guidance  system  with  the  short-range  field  generator 
is  similar  to  the  Aurora  system.  When  using  the  flat-panel  configuration,  the  field  generator  is  placed  underneath 
the  patient  (Figure  1(d)).  This  requires  that  the  standard  mattress  be  replaced  with  one  which  will  support  the 
patient.  This  is  due  to  the  shape  of  the  patient  couch,  which  is  curved  with  the  flat-panel  field  generator  raising 
the  patient  above  the  bed.  This  will  also  cause  problems  with  obese  patients,  as  the  effective  bore  size  is  reduced. 

In  the  needle  biopsy  procedures  there  are  two  modes  of  imaging,  acquisition  of  a  preoperative  CT  scan  and 
intraoperative  CT  fluoroscopy,  real  time  single  slice  imaging.  The  Aurora  and  3D  Guidance  with  short-range 
field  generator  are  not  in  the  field  of  view  when  imaging  and  thus  do  not  effect  image  quality.  When  using  the 
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3D  Guidance  system  with  the  flat-panel  field  generator  the  images  always  included  the  field  generator  as  it  is 
placed  underneath  the  patient.  We  performed  the  same  imaging  experiment  as  in  the  interventional  radiology 
suite,  showing  that  the  image  quality  was  degraded  by  the  field  generator’s  presence.  Detailed  results  are  given 
in  section  3. 

Pulmonology  suite 

In  the  transbronchial  biopsy  procedure  the  patient  is  lying  on  a  strecher  at  a  45°  upright  angle,  with  the 
mechanical  arm  holding  the  Aurora  field  generator  attached  to  the  stretcher’s  rail  (Figure  1(e)).  This  restricts 
physical  access  to  one  side  of  the  patient.  This  is  not  a  limiting  factor  as  the  physician  stands  on  the  opposite 
side  of  the  stretcher  facing  the  bronchoscopy  monitor.  Again,  the  3D  Guidance  system  with  the  short-range  field 
generator  was  similar  to  the  Aurora  system,  and  the  flat-panel  field  generator  is  unobtrusive  as  it  is  placed  under 
the  patient’s  back  (Figure  1(f)). 

None  of  the  systems  caused  any  noticeable  distortions  of  the  video  imaging. 

2.5.  Completeness 

Both  the  Aurora  and  3D  Guidance  system  support  5DOF  and  6DOF  sensors.  For  the  Aurora  system  the  smallest 
5(6)DOF  sensors  have  a  diameter  of  0.55(1.8)rara.  For  the  3D  Guidance  system  the  smallest  5(6)DOF  sensors 
have  a  diameter  of  0.3 (1.3) mm. 

Interventional  radiology  suite 

The  tools  that  require  tracking  in  our  procedures  are  18  gauge  (diameter  of  1.02mm)  needles  for  RFA,  and 
TIPS,  22  gauge  (diameter  of  0.6mm)  needles  for  vertebroplasty,  a  catheter  with  a  diameter  of  2.3 mm  for  stent 
deployment,  and  a  vertebroplasty  trochar  with  a  diameter  of  A. 2mm.  All  of  these  tools  are  similar  in  that  they 
are  used  in  a  way  that  allows  us  to  model  them  as  cylinders.  That  is,  the  rotation  around  the  tool  axis  is 
unimportant.  This  is  why  5DOF  sensors  are  sufficient  for  all  of  these  procedures. 

CT  suite 

The  tools  required  for  the  biopsy  procedures  performed  in  the  CT  suite  are  18  gauge  needles  as  used  in  the 
procedures  performed  in  the  interventional  radiology  suite. 

Pulmonology  suite 

The  tool  that  we  need  to  track  for  the  transbronchial  biopsy  is  a  forcep  that  must  fit  through  the  bronchoscope’s 
working  channel.  In  our  case  the  working  channel  has  a  diameter  of  2 mm,  and  the  forceps  have  a  sheath 
diameter  of  1.8mm.  For  this  procedure  we  cannot  use  the  same  cylindrical  model  as  done  for  other  procedures. 
Navigation  is  performed  using  a  preoperative  CT  in  which  a  virtual  camera  is  positioned  in  the  same  pose  as  the 
bronchscope’s  camera  which  is  tracked  relative  to  the  forceps.  The  tracked  forceps  are  thus  required  to  provide 
all  six  degrees  of  freedom.  As  the  unknown  rotation  around  the  sensor  axis  is  similar  to  an  unknown  camera 
rotation  around  the  view  direction.  While  the  6DOF  sensors  are  close  to  the  sheath  diameter  it  has  been  shown 
that  it  is  possible  to  use  a  5DOF  sensor  to  perform  the  tracking  in  combination  with  video-to-CT  registration 
to  compensate  for  the  unknown  rotation.13 

2.6.  Accuracy 

The  accuracy  of  each  system  was  evaluated  using  the  protocol  described  in.3  A  plexiglass  phantom  is  used  to 
acquire  data  measurements  at  225  locations.  At  each  location,  100  measurements  are  acquired  and  their  average 
is  used  as  the  point’s  coordinates.  We  also  record  the  distance  of  each  of  these  100  samples  from  the  EMTS  origin 
and  the  range  of  the  distance  variability,  reflecting  the  system’s  stability.  We  then  register  the  acquired  point  set 
to  the  phantom’s  coordinate  system  using  paired-point  rigid  registration  and  compute  the  distance  between  the 
known  point  location  and  the  reported  point  location  after  it  is  transformed.  For  the  distance  variability  data 
we  report  on  the  maximal  variability  range: 

max[max{<A}j  —  min {di}j],  i  —  1 . . .  100,  j  —  1 . . .  225 

where  di  is  the  distance  from  the  EMTS  origin  to  the  i’th  point.  For  the  registered  point  set  data  we  provide 
the  following  descriptive  statistics:  RMS  error,  mean  error,  standard  deviation,  error  range,  maximal  error  and 
95  percentile.  This  experiment  was  performed  in  each  of  the  environments  described  in  this  paper.  Detailed 
results  are  described  in  section  3. 
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2.7.  Robustness 
Interventional  radiology  suite 

In  the  interventional  radiology  suite  the  device  that  is  potentially  most  disruptive  to  electromagnetic  tracking 
is  the  C-arm.  To  evaluate  its  effect  on  the  stability  of  the  tracking  we  performed  the  following  experiment.  We 
first  placed  two  tracked  sensors  at  an  unknown  fixed  distance  from  each  other.  We  then  acquired  data  with 
the  C-arm  in  its  home  position  away  from  the  patient  and  tracking  systems.  The  C-arm  was  then  moved  into 
imaging  position,  and  an  image  and  tracking  data  were  simultaneously  acquired.  Finally,  we  acquired  tracking 
data  during  a  cone-beam  CT  scan.  For  each  of  these  data  acquisitions  we  computed  the  distance  between  the 
two  sensors  based  on  their  respective  transformations.  Detailed  results  are  given  in  section  3. 

CT  suite 

In  the  CT  suite  both  the  patient  couch  and  CT  gantry  are  potentially  disruptive  to  electromagnetic  tracking. 
Both  these  components  are  active  while  a  CT  scan  is  acquired.  When  CT  fluoroscopy  is  used  only  the  gantry 
rotates  and  the  couch  remains  stationary.  To  evaluate  the  effect  of  these  elements  on  the  stability  of  the  tracking 
we  performed  the  following  experiment.  We  first  placed  two  tracked  sensors  at  an  unknown  fixed  distance  from 
each  other. We  then  acquired  data  at  the  spatial  location  where  we  expect  to  perform  tracking,  away  from  the 
gantry.  This  was  followed  by  tracking  data  acquisition  during  CT  fluoroscopy  imaging,  and  finally  tracking  data 
acquisition  while  a  CT  scan  was  performed.  For  each  of  the  data  acquisitions  we  computed  the  distance  between 
the  two  sensors  based  on  their  respective  transformations.  Detailed  results  are  given  in  section  3. 

Pulmonology  suite 

In  the  pulmonology  suite  the  potentially  problematic  equipment  is  the  patient’s  stretcher  itself.  In  our  institute 
the  procedure  is  performed  with  the  patient  lying  on  the  stretcher  at  an  angle  of  45°.  Unlike  the  interventional 
radiology  and  CT  suites  where  the  patient  couches  are  fixed,  the  stretcher  model  varies,  as  the  procedure 
is  performed  on  whichever  stretcher  the  patient  is  being  transported  with.  This  may  result  in  performance 
variability  based  on  the  stretcher  type.  In  our  case  we  tested  with  a  single  stretcher  type,  the  Hill- Rom  (Batesville, 
Indiana,  USA)  P8000  transport  stretcher,  and  thus  cannot  provide  comprehensive  conclusions. 

3.  EXPERIMENTAL  RESULTS 

We  now  describe  our  experimental  results  with  regard  to  the  effect  the  EMTS  have  on  imaging,  and  our  accuracy 
and  robustness  evaluations  of  these  systems. 

3.1.  Effect  on  imaging 

To  assess  the  effect  of  the  electromagnetic  tracking  systems  on  imaging  we  acquired  images  while  tracking.  In 
the  interventional  radiology  suite  we  acquired  cone-beam  CT  and  projection  images,  in  the  CT  suite  we  acquired 
a  CT  scan  and  CT-fluoroscopy,  and  in  the  pulmonology  suite  we  acquired  video  images. 

In  all  suites  the  Aurora  and  3D  Guidance  with  short-range  field  generator  had  no  noticeable  effect  on  image 
quality,  as  they  were  easily  moved  away  from  the  imaged  region.  The  3D  Guidance  with  flat-panel  field  generator 
did  result  in  a  noticeable  degradation  in  image  quality  when  acquiring  cone-beam  CT  data,  projection  X-ray 
images,  CT-fluoroscopy  and  standard  CT  data.  The  main  cause  for  the  loss  in  image  quality  is  the  presence  of 
the  field  generator  in  the  imaged  region.  As  these  images  are  used  in  subjective  interpretation  of  the  anatomical 
structures,  we  cannot  determine  if  the  lower  image  quality  precludes  the  use  of  the  flat-panel  field  generator. 
This  is  left  up  to  the  physician’s  discretion.  Figure  4  summarizes  these  experiments. 

3.2.  Accuracy 

The  accuracy  of  each  of  the  tracking  systems  was  evaluated  as  described  above.  The  interventional  radiology 
suite  was  found  to  be  the  optimal  environment  for  tracking.  This  is  the  only  environment  that  can  be  controlled 
so  that  the  distortion  of  the  EMTS  magnetic  field  is  minimal.  The  C-arm,  which  is  a  potential  cause  for  distortion 
can  be  placed  in  its  home  position  far  from  the  patient.  In  addition,  the  region  of  the  patient  table  over  which 
the  EMTS  work  volume  is  positioned  is  suspended  in  mid-air.  Table  1  summarizes  the  accuracy  evaluation  in 
the  interventional  radiology  suite.  The  3D  Guidance  with  short-range  field  generator  was  the  most  accurate  in 
this  environment,  with  the  Aurora  and  3D  Guidance  with  flat-panel  field  generator  having  comparable  accuracy. 
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Figure  4.  Effect  of  flat-panel  field  generator  on  image  quality,  all  images  are  of  the  CIRS  abdominal  phantom.  Notice 
deteriorated  image  quality  in  marked  regions.  (a,b)  X-ray  projection  images  without  and  with  field  generator.  (c,d)  Cone- 
beam  CT  axial  slice  without  and  with  field  generator.  (e,f)  CT  axial  slice  without  and  with  field  generator.  CT-fluoroscopy 
images  were  similar  to  images  (e,f). 
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On  the  other  hand  the  3D  Guidance  with  short-range  field  generator  was  the  worst  in  terms  of  stability.  That 
is,  measurements  fluctuated  considerably  but  when  averaged  gave  excellent  accuracy. 


Aurora 

3D  Guidance  short-range 

3D  Guidance  flat-panel 

maximal  sample  distance  variability 

0.26 

4.48 

0.67 

RMS 

1.01 

0.38 

0.77 

mean 

0.76 

0.34 

0.61 

standard  deviation 

0.67 

0.18 

0.48 

range 

3.87 

0.87 

2.30 

max 

3.91 

0.90 

2.39 

95  percentile 

2.20 

0.69 

1.91 

Table  1.  Results  of  accuracy  experiments  in  interventional  radiology  suite.  All  measurements  are  in  millimeters. 

In  the  CT  suite  there  are  two  primary  causes  of  electromagnetic  distortion,  the  patient  couch  and  the  CT 
gantry.  In  practice,  navigation  is  performed  as  far  as  possible  from  the  CT  gantry,  so  that  the  couch  is  the 
primary  cause  of  distortion.  Table  2  summarizes  the  accuracy  evaluation  in  the  CT  suite.  In  this  environment 
the  3D  Guidance  with  flat-panel  was  the  most  accurate.  Note  that  the  main  cause  of  the  distortion  is  the 
couch  mechanism  that  is  underneath  the  field  generator,  which  complies  with  the  system’s  assumptions.  The  3D 
Guidance  with  short-range  field  generator  and  the  Aurora  system  had  comparable  accuracy. 


Aurora 

3D  Guidance  short-range 

3D  Guidance  flat-panel 

maximal  sample  distance  variability 

0.54 

2.54 

0.30 

RMS 

5.76 

6.49 

1.08 

mean 

5.14 

5.67 

1.02 

standard  deviation 

2.62 

3.17 

0.36 

range 

18.33 

17.34 

1.76 

max 

19.24 

18.14 

1.96 

95  percentile 

10.83 

12.29 

1.56 

Table  2.  Results  of  accuracy  experiments  in  CT  suite.  All  measurements  are  in  millimeters. 

In  the  pulmonology  suite  the  primary  source  of  distortion  is  the  patient  stretcher.  In  our  case  we  evaluated 
accuracy  with  the  Hill-Rom  (Batesville,  Indiana,  USA)  P8000  transport  stretcher.  Table  3  summarizes  the 
accuracy  evaluation  in  the  pulmonology  suite.  In  this  environment  the  Aurora  and  3D  Guidance  with  short-field 
generator  were  comparable  and  more  accurate  than  the  3D  Guidance  with  flat-panel  field  generator. 


Aurora 

3D  Guidance  short-range 

3D  Guidance  flat-panel 

maximal  sample  distance  variability 

0.26 

1.85 

2.43 

RMS 

1.16 

1.00 

3.14 

mean 

0.95 

0.89 

2.78 

standard  deviation 

0.67 

0.45 

1.46 

range 

5.70 

3.76 

10.02 

max 

5.83 

3.92 

10.40 

95  percentile 

2.19 

1.54 

5.78 

Table  3.  Results  of  accuracy  experiments  in  pulmonology  suite.  All  measurements  are  in  millimeters. 


3.3.  Robustness 

Robustness  of  the  EMTS  was  evaluated  in  the  CT  and  interventional  radiology  suites  as  described  in  section  2.7. 

In  the  CT  suite  when  acquiring  data  far  from  the  gantry  the  mean(std)  distance  between  the  two  tracked  sen¬ 
sors  for  the  Aurora  system  was  46. 73 (0.04) mm,  for  the  3D  Guidance  with  flat-panel  field  generator  the  mean(std) 
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distance  was  55.82 (O.lO)mra  and  with  the  short-range  field  generator  58.03(0.38)ram.  We  then  acquired  tracking 
data  during  acquisition  of  a  single  CT-fluorscopy  image  and  during  a  CT  scan.  For  the  Aurora  system  30%  of  the 
data  was  not  reported  for  the  single  image  case,  and  93.4%  of  the  data  during  the  CT  scan.  Figure  5  summarizes 
the  data  acquired  using  the  3D  Guidance  system  during  single  image  acquisition  and  during  a  complete  CT  scan. 
Based  on  these  results  we  conclude  that  all  three  systems  cannot  be  used  during  image  acquisition. 


(a)  (b) 
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Figure  5.  Distances  between  two  fixed  sensors  during  single  CT-fluoroscopy  image  acquisition  (left  column)  and  during 
CT  scan  (right  column).  (a,b)  3D  Guidance  with  short-range  field  generator,  and  (c,d)  3D  Guidance  with  flat-panel  field 
generator. 


In  the  interventional  radiology  suite  when  acquiring  data  with  the  C-arm  away  from  the  tracking  location 
the  mean(std)  distance  between  the  two  tracked  sensors  for  the  Aurora  system  was  21.84(0.06)mm,  for  the  3D 
Guidance  with  flat-panel  field  generator  the  mean(std)  distance  was  52.28(0. 12)mm  and  with  the  short-range 
field  generator  52.63(0.61  )mm.  Note  that  for  each  system  the  distance  can  be  different  as  the  sensors  are  placed  in 
an  arbitrary  position.  Figure  6  summarizes  this  experiment,  with  data  for  each  of  the  systems  as  recorded  during 
a  single  fluorscopic  image  acquisition  and  during  a  rotation  of  the  C-arm  for  cone-beam  CT  data  acquisition. 
Based  on  our  results  we  conclude  that  the  Aurora  and  3D  Guidance  system  with  short-range  field  generator 
should  not  be  used  in  conjunction  with  imaging,  and  the  3D  Guidance  system  with  flat-panel  field  generator  can 
be  used  during  imaging,  although  it  does  have  a  detrimental  effect  on  the  images  as  described  above. 


4.  DISCUSSION  AND  CONCLUSIONS 

We  have  presented  a  holistic  approach  to  evaluating  electromagnetic  tracking  systems  in  the  clinical  environment 
that  takes  into  account  requirements  beyond  measurement  accuracy. 

In  the  past  electromagnetic  tracking  system  evaluation  has  focused  on  the  system’s  accuracy  and  ability  to 
deal  with  the  presence  of  electromagnetic  field  distorting  objects.  This  type  of  assessment  is  insufficient  for  the 
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Figure  6.  Distances  between  two  fixed  sensors  during  single  X-ray  image  acquisition  (left  column)  and  during  C-arm 
rotation  for  cone-beam  CT  data  acquisition  (right  column).  (a,b)  Aurora  system,  (c,d)  3D  Guidance  with  short-range 
field  generator,  and  (e,f)  3D  Guidance  with  flat-panel  field  generator. 


clinical  environment.  For  example,  the  3D  Guidance  system  using  the  flat-panel  configuration  was  found  to  be 
the  most  accurate  in  the  CT  suite  environment,  but  because  it  is  placed  underneath  the  patient  it  degrades  the 
quality  of  the  CT  and  CT-fluoroscopy  images.  Thus  if  accuracy  were  the  only  requirement  this  would  be  the 
system  of  choice.  Given  that  imaging  is  at  the  heart  of  these  interventions  it  is  up  to  the  physician  to  decide  if 
the  quality  of  the  images  is  sufficient. 

Based  on  our  experience  using  electromagnetic  tracking  in  the  clinical  environment  we  have  compiled  a 
comprehensive  list  of  requirements.  These  requirements  should  be  assessed  on  a  per-procedure  basis  and  include, 
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the  system’s  refresh  rate,  the  number  of  sensors  that  need  to  be  tracked,  the  size  of  the  navigated  region,  system 
interaction  with  the  environment,  can  the  sensors  be  embedded  into  the  tools  and  provide  the  desired  information 
(5DOF  vs.  6DOF),  and  tracking  accuracy  and  robustness  in  the  specific  interventional  environment.  We  assessed 
the  Aurora  EMTS,  and  the  3D  Guidance  EMTS  with  short-range  and  flat-panel  field  generator  configurations 
using  these  requirements.  Each  of  the  systems  was  found  to  have  certain  strengths  and  weaknesses,  but  none  of 
them  was  optimal  across  environments  and  procedures. 
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Abstract  Positron-emission  tomography  (PET)  and  PET/CT 
(computed  tomography)  are  becoming  increasingly  important  for 
diagnosis  and  treatment  of  cancer.  Clinically  relevant  changes  can 
sometimes  be  seen  on  PET  that  are  not  seen  on  other  imaging 
modalities.  However,  PET  is  not  suitable  for  guiding  biopsy  as  the 
images  are  not  obtained  in  real-time.  Therefore,  our  research 
group  has  begun  developing  a  concept  for  PET/CT-guided  inter¬ 
ventional  procedures.  This  paper  presents  the  rationale  for  this 
concept,  outlines  our  research  plan,  and  includes  an  initial  study 
to  evaluate  the  relative  sensitivity  of  CT  and  PET /CT  in  detecting 
suspicious  lesions. 

Keywords  PET  •  PET/CT  •  Image  guidance  •  Cancer  • 
Interventional  procedures 

1.  Introduction 

In  recent  years,  PET  (positron-emission  tomography)  and  PET/CT 
(computed  tomography)  have  become  important  tools  in  cancer 
management  and  treatment  [2].  PET  can  provide  information  on 
functional  or  metabolic  characteristics  of  tumors,  whereas  CT  and 
MRI  (magnetic  resonance  imaging)  predominately  assess  the  tu¬ 
mor’s  anatomical  and  morphological  features,  such  as  density, 
size,  and  shape.  PET  can  sometimes  detect  clinically  relevant 
changes  that  are  not  seen  on  CT  or  MRI. 

Given  this  rising  importance  of  PET  imaging  in  the  staging  of 
cancer,  our  research  group  has  begun  to  develop  a  methodology 
for  PET-guided  interventional  procedures.  Our  goal  is  to  combine 
respiratory  corrected  PET  images  with  rotational  angiography 
images  in  the  interventional  suite  for  precision  image-guided 
biopsy.  This  paper  describes  some  of  our  initial  work  in  this  area, 
including  some  background  on  image-guided  biopsy,  example 
images,  and  an  initial  study  of  the  sensitivity  of  CT  alone  versus 
PET/CT  for  detecting  malignant  lesions. 

2.  Methods 

2.1.  Image-guided  biopsy 

Although  biopsy  remains  the  gold  standard  for  diagnosis  and 
identification  of  disease,  it  still  has  several  key  limitations.  First, 

tumors  can  often  be  heterogeneous  in  composition,  requiring 
multiple  sampling  attempts  to  get  sufficient  tissue  for  pathological 
evaluation.  Second,  as  tumors  are  being  detected  at  earlier  and 
earlier  stages,  the  biopsy  needle  must  hit  smaller  and  smaller 
targets,  which  can  be  quite  challenging  when  directing  a  percu¬ 
taneous  needle.  Third,  percutaneous  biopsy  is  often  performed 
using  solely  X-ray  guidance,  which  is  limited  to  showing  structural 
details  of  the  anatomy  such  as  bones,  organ  boundaries,  and  the 


circulatory  system.  Finally,  even  when  a  diagnostic  PET/CT  is 
available,  it  is  typically  only  available  on  a  separate  computer 
workstation,  forcing  the  physician  to  mentally  integrate  and 
register  the  images.  Although  many  physicians  are  quite  skilled  at 
this  process,  it  is  challenging  and  potentially  subject  to  significant 
errors,  as  shown  in  Fig.  1  below. 

2.2.  Minimally  invasive  tumor  ablation  therapies 

Minimally  invasive  therapies,  particularly  radiofrequency  ablation 
(RFA),  are  seeing  increasing  use  because  of  their  ability  to  treat 
tumors  effectively  with  minimal  side  effects  and  relatively  short 
recovery  times  [1,  3,  4].  Certain  populations  of  patients,  such  as 
those  for  whom  open  surgery  is  contraindicated,  may  find  that  a 
minimally  invasive  approach  is  their  only  treatment  option.  PET- 
guided  interventional  radiology  is  an  ideal  technology  partner  for 
minimally  invasive  therapy.  Minimally  invasive  therapy  allows 
physicians  to  precisely  target  small  regions  of  the  anatomy,  and 
PET  images  provide  the  information  necessary  to  treat  only  the 
functionally  active  lesion. 

It  is  well  known  that  tumors  are  heterogeneous  and  that  the 
different  regions  of  the  tumor  are  often  not  readily  apparent  on 
structural  imaging  such  as  CT  or  MRI.  Based  on  our  clinical 
experience,  we  can  immediately  identify  two  strong  impacts  of 
PET-guided  interventional  radiology  therapies.  First,  as  tumors 
outgrow  their  blood  supply,  portions  of  the  tumor  may  become 
necrotic.  PET  information  would  be  useful  to  identify  the  viable 
fraction  of  specific  tumor(s)  for  targeted  therapy.  Second,  many 
tumors  respond  partially  to  chemotherapy,  so  PET  imaging 
would  be  useful  to  identify  chemo-resistant  tumor  tissue  for 
second-line  or  salvage  therapy  using  a  minimally  invasive  ap¬ 
proach. 

2.3.  Challenges  to  combining  PET/CT  and  interventional 
radiology 

Despite  the  significant  capabilities  of  PET /CT  imaging,  clinically 
viable  solutions  for  PET-guided  interventional  radiology  have  not 
yet  been  developed.  Obviously,  the  most  direct  method  for 
accomplishing  PET  guided  interventional  radiology  would  be  to 
perform  the  intervention  with  the  patient  on  the  PET /CT  scanner 
table.  Our  clinical  partners  at  Georgetown  University  Hospital 
have  in  fact  done  a  small  number  of  biopsy  procedures  on  our 
current  scanner.  However,  this  early  work  has  uncovered  several 
reasons  why  this  solution  is  impractical  and  unlikely  to  be  widely 
implemented. 

1.  PET  and  CT  scanners  are  designed  for  producing  diagnostic 
snapshot  images  of  a  patient,  rather  than  the  sustained  real 
time  imaging  required  for  image  guidance. 


Fig.  1  CT  image  (left)  and  PET/CT  fused  image  (right).  With 
CT  alone,  the  tumor  in  the  liver  is  inconspicuous,  but  with  PET/ 
CT  the  tumor  is  easily  seen  as  the  bright  red  spot  indicated  by 
the  crosshairs  (there  is  also  a  second  lesion  in  the  kidney  at  lower 
right  on  the  PET/CT  image).  These  images  clearly  illustrate  the 
challenge  of  attempting  image-guided  biopsy  using  only  CT  or 
X-ray  fluoroscopy  for  navigation 
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2.  Images  from  the  PET/CT  scanner  are  displayed  on  worksta¬ 
tions  located  outside  the  room  with  the  scanner  itself,  so  using 
these  workstations  for  guidance  involves  a  lot  of  tedious 
movement  back  and  forth  from  the  patient  table  to  the  display 
workstation. 

3.  Infrastructure  requirements  for  an  interventional  radiology 
suite  (e.g.,  a  large  number  of  complex  instruments  and  the 
ability  to  maintain  a  sterile  field  around  the  patient)  are  very 
different  from  those  of  a  CT  scanner  suite,  necessitating  either 
a  duplication  of  equipment  or  laborious  relocation  of  that 
equipment  during  procedures. 

4.  Even  on  CT  systems  that  support  a  fluoroscopic  mode,  the 
utilization  time  for  interventions  remains  a  major  financial 
impediment.  The  time  required  for  a  successful  intervention 
(including  setup,  the  procedure,  and  cleanup)  can  extend  into 
multiple  hours,  and  therefore  significantly  decreases  the  nor¬ 
mal  billable  imaging  time  for  the  system. 

2.4.  Clinical  rationale  and  preliminary  study 
To  investigate  the  clinical  rationale  for  developing  this  system,  we 
completed  a  preliminary  study  to  evaluate  the  relative  sensitivity 
of  CT  and  PET/CT  in  detecting  lesions  suspicious  for  malignancy 
in  whole  body  studies.  A  retrospective  analysis  of  53  consecutive 
whole  body  FDG  PET/CT  studies  with  a  total  of  164  lesions  was 
completed.  Negative  studies  and  studies  with  only  head  or  neck 
findings  were  excluded.  A  single  reader  retrospectively  analyzed 
first  the  CT  data  alone,  then  the  fused  PET/CT  images.  PET 
positive  lesions  were  categorized  based  on  CT  appearance  in 
relation  to  possible  malignancy.  Six  categories  of  CT  findings  were 
defined.  Grade  0:  no  lesion  on  CT;  grade  1:  benign-appearing 
lesion  on  CT;  grade  2:  suspicious  lesion  on  CT;  grade  3:  definite 
malignancy  on  CT;  grade  R/0:  benign  appearing  CT  lesion  seen 
only  in  retrospect;  grade  R/l:  suspicious  appearing  CT  lesion  seen 
only  in  retrospect.  Grade  0,  1,  R/0  and  R/l  were  classified  as 
“discordant,”  in  that  the  CT  data  alone  did  not  identify  a  PET/CT 
positive  lesion. 

3.  Results 

Of  the  164  reviewed  lesions,  the  total  discordant  number  is  68,  or 
41.5%.  Of  the  42  grade  0  lesions  (25.6%  of  the  total),  16  were 
located  in  bone,  12  lymph  nodes,  6  soft  tissues,  3  gastrointestinal 
track,  and  1  each  in  the  breast,  lung,  vulva,  cervix,  and  prostate. 
Of  the  18  grade  1  lesions  (11.0%  of  the  total),  7  were  located  in 
lymph  nodes,  4  soft  tissues,  and  1  each  in  both  lung  and  bone.  Of 
the  6  R/0  lesions  (3.7%  of  the  total),  2  were  located  in  the  lung, 
and  1  in  each  the  liver,  lymph  node,  pelvic  mass,  and  soft  tissue. 
Of  the  4  R/l  lesions  (2.4%  of  the  total),  2  were  located  in  the  lung, 
and  one  in  each  the  rectum  and  bone.  There  were  also  30  grade  2 


lesions  (18.3%  of  the  total),  and  62  grade  3  lesions  (37.8%  of  the 
total). 

The  conclusion  of  the  preliminary  study  was  thus  that  CT  alone 
underestimates  41.5%  of  PET/CT  positive  cases  of  whole  body 
scans  for  the  detection  of  malignancy.  Soft  tissue  and  lymph  node 
lesions  were  the  lesions  most  likely  to  be  missed  by  CT  alone. 

4.  Conclusions  and  summary 

Considering  these  potential  applications  and  the  challenges  posed 
by  the  existing  state-of-the-art,  we  believe  that  the  best  course  is  to 
devise  a  means  for  integrating  the  PET/CT  information  into  the 
interventional  radiology  workspace.  This  approach  allows  us  to 
optimize  the  acquisition  of  PET/CT  images  and  the  actual  inter¬ 
vention  on  the  patient  separately,  thereby  avoiding  the  compro¬ 
mises  required  by  trying  to  perform  all  of  these  procedures  on  a 
single  piece  of  equipment. 

Our  work  will  provide  accurate  registration  of  the  PET/CT  images 
into  the  interventional  radiology  coordinate  system,  making  the 
functional  data  readily  accessible  at  the  time  of  the  intervention. 
In  summary,  this  project  will  combine  leading-edge  technologies 
in  PET/CT,  image  registration,  and  interventional  radiology  to 
address  the  pressing  clinical  needs  of  image-guided  biopsy  and 
minimally  invasive  therapies. 
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